Slrpnk.net outage
-
This is a very big issue for a server no mater when it happens. A sever you can’t reset and you go on a trip ridiculous.
It usually can be reset, it's entirely unexpected that a component that enables that functionality would randomly break. You're essentially implying that any admin who self hosts cannot ever leave their house for any length of time without being irresponsible because they cannot ensure with 100% certainly that a computer component won't fail while their gone, or a lightning storm won't fry something, or a flood won't harm the servers.
Like, dude, c'mon. We're volunteers, and poVoq is running this stuff for free. We're not an enterprise business with multiple staff on hand to swap out failed components 24/7 so that our 'customers' don't lose millions of dollars.
If anything, this only proves the strength of decentralized media, as we're able to swap to a functioning node and continue on our merry way while the other is being repaired. Like what an incredible and resilient system considering none of this is making anyone any money.
-
Alt account of poVoq@slrpnk.net here.
Our instance is currently down and I can't get remote access to the servers. It appears that there might have been a hardware failure of the main firewall, which is the one thing I can't work around remotely.
I am still trying a few things, but I am not very optimistic that I can get access.
The really unfortunate part is that just now I am on one of my rare work deployments abroad, so I also can't access it physically during the next few weeks and my usual back up that could restart it is not available either.
As something like that never happened in 3 years operating the servers, I thought I can risk it, but murphy's law seems inescapable
I will try to keep you posted here on any updates, but probably there will not be much I can do for a while. Really bad timing
Edit: we might use this "opportunity" to migrate the instance to Piefed, which has been an idea for quite some time now. I will keep you posted on that.
Shit happens. Thanks for the update
-
Alt account of poVoq@slrpnk.net here.
Our instance is currently down and I can't get remote access to the servers. It appears that there might have been a hardware failure of the main firewall, which is the one thing I can't work around remotely.
I am still trying a few things, but I am not very optimistic that I can get access.
The really unfortunate part is that just now I am on one of my rare work deployments abroad, so I also can't access it physically during the next few weeks and my usual back up that could restart it is not available either.
As something like that never happened in 3 years operating the servers, I thought I can risk it, but murphy's law seems inescapable
I will try to keep you posted here on any updates, but probably there will not be much I can do for a while. Really bad timing
Edit: we might use this "opportunity" to migrate the instance to Piefed, which has been an idea for quite some time now. I will keep you posted on that.
Sending positive vibes, and passing offer of technical assistance once you get access again.
Hopefully it's an easy fix once you get into things again.
-
Well, on the plus side lots of lessons learned and I think I might move at least the xmpp server to an external vps to have a backup communication channel.
Not sure about your area, but a wireguard accessible OOB connection is a great piece of kit to keep handy. I use a cheap 768kbps SIM in an Ethernet connected switch into my personal systems. It's saved my skin numerous times.
I'm sure this is obvious, especially in hindsight, but just mentioning because the existence of IoT LTE data plans for a minimal fee ( $100/year for me in Midwest US) was NOT obvious to me until 2 years ago.
-
Seriously, I think a big part of solarpunk ethos is combating the notion that everything has to always be available 24/7. Society pays a lot to deliver every convenience like fruit out of season from the other side of the world.
And on the punk side of things, anybody familiar with volunteer community action, cooperative events, whatever, knows that sometimes a person doing shit for the community fails to do the shit they were supposed to do because some other shit happened in their life. It happens. We just move on to the backup shit-doer-person or find a workaround for the lack of shit.
And this is the cool thing about a distributed network like Lemmy. If a giant centralized database like FB goes down, its users can't do anything. If our favorite Lemmy node goes down, we can log into other nodes until our home base is back up.
-
Not sure about your area, but a wireguard accessible OOB connection is a great piece of kit to keep handy. I use a cheap 768kbps SIM in an Ethernet connected switch into my personal systems. It's saved my skin numerous times.
I'm sure this is obvious, especially in hindsight, but just mentioning because the existence of IoT LTE data plans for a minimal fee ( $100/year for me in Midwest US) was NOT obvious to me until 2 years ago.
Yeah I had plans to set up something like that, but always other priorities and in this specific case I could maybe access other internal servers but i would need KVM access to reboot the firewall or some other way to cut physical power. And exfiltrating hundreds of GBs of lemmy database wouldn't work over such a small pipe either.
-
Alt account of poVoq@slrpnk.net here.
Our instance is currently down and I can't get remote access to the servers. It appears that there might have been a hardware failure of the main firewall, which is the one thing I can't work around remotely.
I am still trying a few things, but I am not very optimistic that I can get access.
The really unfortunate part is that just now I am on one of my rare work deployments abroad, so I also can't access it physically during the next few weeks and my usual back up that could restart it is not available either.
As something like that never happened in 3 years operating the servers, I thought I can risk it, but murphy's law seems inescapable
I will try to keep you posted here on any updates, but probably there will not be much I can do for a while. Really bad timing
Edit: we might use this "opportunity" to migrate the instance to Piefed, which has been an idea for quite some time now. I will keep you posted on that.
That's unlucky, but is it hosted at someone's home?
-
Alt account of poVoq@slrpnk.net here.
Our instance is currently down and I can't get remote access to the servers. It appears that there might have been a hardware failure of the main firewall, which is the one thing I can't work around remotely.
I am still trying a few things, but I am not very optimistic that I can get access.
The really unfortunate part is that just now I am on one of my rare work deployments abroad, so I also can't access it physically during the next few weeks and my usual back up that could restart it is not available either.
As something like that never happened in 3 years operating the servers, I thought I can risk it, but murphy's law seems inescapable
I will try to keep you posted here on any updates, but probably there will not be much I can do for a while. Really bad timing
Edit: we might use this "opportunity" to migrate the instance to Piefed, which has been an idea for quite some time now. I will keep you posted on that.
Good luck and all the best. Time to touch grass, everyone
-
That's unlucky, but is it hosted at someone's home?
Sort of yes. It is a homelab a few people have access to, but currently they are all either on holiday or on extended work related travel. It seemed like an acceptable risk as everything worked smoothly for years.
-
This is a very big issue for a server no mater when it happens. A sever you can’t reset and you go on a trip ridiculous.
You're going to want to try "my little Internet" by Fisher-Price before using this adult Internet we're on.
-
Sort of yes. It is a homelab a few people have access to, but currently they are all either on holiday or on extended work related travel. It seemed like an acceptable risk as everything worked smoothly for years.
That makes sense, given the ethos of your instance, I hope you can fix it a soon as reason allows!
-
Alt account of poVoq@slrpnk.net here.
Our instance is currently down and I can't get remote access to the servers. It appears that there might have been a hardware failure of the main firewall, which is the one thing I can't work around remotely.
I am still trying a few things, but I am not very optimistic that I can get access.
The really unfortunate part is that just now I am on one of my rare work deployments abroad, so I also can't access it physically during the next few weeks and my usual back up that could restart it is not available either.
As something like that never happened in 3 years operating the servers, I thought I can risk it, but murphy's law seems inescapable
I will try to keep you posted here on any updates, but probably there will not be much I can do for a while. Really bad timing
Edit: we might use this "opportunity" to migrate the instance to Piefed, which has been an idea for quite some time now. I will keep you posted on that.
That sucks. Thank you for your time and running the instance. Gonna miss you all like hell while you're gone.
-
Alt account of poVoq@slrpnk.net here.
Our instance is currently down and I can't get remote access to the servers. It appears that there might have been a hardware failure of the main firewall, which is the one thing I can't work around remotely.
I am still trying a few things, but I am not very optimistic that I can get access.
The really unfortunate part is that just now I am on one of my rare work deployments abroad, so I also can't access it physically during the next few weeks and my usual back up that could restart it is not available either.
As something like that never happened in 3 years operating the servers, I thought I can risk it, but murphy's law seems inescapable
I will try to keep you posted here on any updates, but probably there will not be much I can do for a while. Really bad timing
Edit: we might use this "opportunity" to migrate the instance to Piefed, which has been an idea for quite some time now. I will keep you posted on that.
Thanks for the info, a healthy amount of real life awaits then.
I knew it was running on solar energy and old hardware so I guessed something like this had happened.
If you need fail-over to awaken a backup system when the primary fails, things can be designed.
-
I know, I'm the one who approved your application.
You happened to sign up right before we've had our first particularly bad issue in 3 years. Unlucky, yes, but it's a bit out of order (IMO) to say we're a poorly managed instance based on that, especially seeing as you lost nothing besides the time it took to write your application.
![😕](https://forum.appy.cat/assets/plugins/nodebb-plugin-emoji/emoji/android/1f615.png?v=457f2f31b39 ":\")
Does Lemmy let you retroactively deny applications?
-
Does Lemmy let you retroactively deny applications?
As a matter of fact it does, actually
-
Alt account of poVoq@slrpnk.net here.
Our instance is currently down and I can't get remote access to the servers. It appears that there might have been a hardware failure of the main firewall, which is the one thing I can't work around remotely.
I am still trying a few things, but I am not very optimistic that I can get access.
The really unfortunate part is that just now I am on one of my rare work deployments abroad, so I also can't access it physically during the next few weeks and my usual back up that could restart it is not available either.
As something like that never happened in 3 years operating the servers, I thought I can risk it, but murphy's law seems inescapable
I will try to keep you posted here on any updates, but probably there will not be much I can do for a while. Really bad timing
Edit: we might use this "opportunity" to migrate the instance to Piefed, which has been an idea for quite some time now. I will keep you posted on that.
Sorry to hear that! My main account is on slrpnk.net. But don't let it stress you out, "solarpunk" says "punk" which implies "DIY" which means this sort of thing happens. Much better to have an instance run by an admin who truly cares, rather than some soulless corporation.
-
Alt account of poVoq@slrpnk.net here.
Our instance is currently down and I can't get remote access to the servers. It appears that there might have been a hardware failure of the main firewall, which is the one thing I can't work around remotely.
I am still trying a few things, but I am not very optimistic that I can get access.
The really unfortunate part is that just now I am on one of my rare work deployments abroad, so I also can't access it physically during the next few weeks and my usual back up that could restart it is not available either.
As something like that never happened in 3 years operating the servers, I thought I can risk it, but murphy's law seems inescapable
I will try to keep you posted here on any updates, but probably there will not be much I can do for a while. Really bad timing
Edit: we might use this "opportunity" to migrate the instance to Piefed, which has been an idea for quite some time now. I will keep you posted on that.
Ty for the update! slrpnk is my favorite instance that isn't my home instance
-
Alt account of poVoq@slrpnk.net here.
Our instance is currently down and I can't get remote access to the servers. It appears that there might have been a hardware failure of the main firewall, which is the one thing I can't work around remotely.
I am still trying a few things, but I am not very optimistic that I can get access.
The really unfortunate part is that just now I am on one of my rare work deployments abroad, so I also can't access it physically during the next few weeks and my usual back up that could restart it is not available either.
As something like that never happened in 3 years operating the servers, I thought I can risk it, but murphy's law seems inescapable
I will try to keep you posted here on any updates, but probably there will not be much I can do for a while. Really bad timing
Edit: we might use this "opportunity" to migrate the instance to Piefed, which has been an idea for quite some time now. I will keep you posted on that.
This is making me want to create an alt on slrpnk.net , since you're so chill and communicative about issues.
-
Alt account of poVoq@slrpnk.net here.
Our instance is currently down and I can't get remote access to the servers. It appears that there might have been a hardware failure of the main firewall, which is the one thing I can't work around remotely.
I am still trying a few things, but I am not very optimistic that I can get access.
The really unfortunate part is that just now I am on one of my rare work deployments abroad, so I also can't access it physically during the next few weeks and my usual back up that could restart it is not available either.
As something like that never happened in 3 years operating the servers, I thought I can risk it, but murphy's law seems inescapable
I will try to keep you posted here on any updates, but probably there will not be much I can do for a while. Really bad timing
Edit: we might use this "opportunity" to migrate the instance to Piefed, which has been an idea for quite some time now. I will keep you posted on that.
Hey fellow slrpnk.net refugees... if you're looking to recreate your list of subscribed communities, here's a good place to start:
::: spoiler click here for a list of communities that are NOT politics, tech, or meme -related.
Most are currently active (except for the ones with a * which were less active last I checked) Sometimes politics, tech or memes sneak in but they're not the focus.
GENERAL DISCUSSION / QUESTIONS
- !ask@lemm.ee
- !asklemmy@lemmy.world
- !askmeanything@lemmy.ca
- !casualconversation@lemm.ee
- !casualuk@feddit.uk
- !Dullsters@dullsters.net
- !dull_mens_club@lemmy.world
- !forumlibre@jlai.lu
- !general@lemmy.world
- !nostupidquestions@lemmy.world
- !goodoffmychest@lemmy.world
- !preguntalealemmy@feddit.cl
- !rant@lemmy.sdf.org
ART / PHOTOS
- !artporn@lemm.ee
- !astrophotography@lemmy.world
- !accidentalrenaissance@lemmy.blahaj.zone
- !badrealestate@feddit.uk *
- !collage@lemmy.ml *
- !desire_paths@sh.itjust.works
- !fakehistoryporn@lemmy.world *
- !fediclicks@viewfinder.pro *
- !filmsframes@lemmy.world
- !foodphotos@sh.itjust.works *
- !gameart@sopuli.xyz
- !grasweeti@slrpnk.net
- !hmmm@lemmy.world
- !imadethis@lemm.ee
- !imaginary@reddthat.com
- !inhabitedbeauty@lemmy.world
- !liminalspace@lemmy.world
- !mobile_wallpaper@lemmy.dbzer0.com
- !nocontextpics@lemmy.world
- !pics@lemmy.world
- !pictures@lemm.ee
- !pixelart@retrolemmy.com
- !pixelart@lemmy.ml *
- !sillydrawingrequests@sopuli.xyz
- !traditional_art@lemmy.world
- !trams_trolleys_streetcars@lemmy.blahaj.zone
- !wallpapers@lemmy.ca
- !52weeksofart@lemmy.world
ANIMALS
- !bats@lemmy.world
- !catposting@lemmy.blahaj.zone
- !cat@lemmy.world
- !catsstandingup@lemmy.world *
- !opossums@lemmy.world
- !scrungycats@lemmy.world *
- !superbowl@lemmy.world
COMICS / GRAPHIC NOVELS
- !alanmoore@feddit.uk *
- !comics@lemmy.blahaj.zone
- !comicstrips@lemmy.world
- !eurographicnovels@lemm.ee
- !truecomics@midwest.social *
- !garfield@lemmy.world
- !outofcontextcomics@lemmy.world
- !unix_surrealism@lemmy.sdf.org
ENTERTAINMENT
- !80smusic@lemmy.world
- !90smusic@lemmy.world
- !90s_tv@lemmy.sdf.org
- !badmovies@lemmy.world
- !bmoviebonanza@lemmy.world
- !Bside@fedia.io
- !boardgames@sopuli.xyz
- !cultfilms@lemux.minnix.dev *
- !dwarffortress@lemmy.ml
- !edm@reddthat.com
- !fullmoviesonyoutube@lemm.ee
- !iasip@sh.itjust.works
- !martialartsmovies@lemm.ee *
- !metal@lemmy.world
- !movies@lemm.ee
- !music@lemmy.world
- !pokemon@lemm.ee
- !pokemon@lemmy.world
- !pokemongo@lemmy.world *
- !ska@lemmy.blahaj.zone
- !thrashmetal@lemmy.world
- !trailers@lemmy.blahaj.zone
GENRES / STYLES
- !cassettefuturism@lemm.ee
- !cosmichorror@lemm.ee
- !cyberpunk@lemmy.zip
- !fuckyournelsonlamp@lemmy.world *
- !goblincore@lemmy.blahaj.zone
- !gothindustrial@lemmy.world
- !lovecraft_mythos@lemmy.world
- !mullets@lemmy.ca *
- !punk@lemmy.world
- !punk_rock@lemmy.ca
- !raygungothic@lemm.ee
- !streetwear@lemmy.world *
- !vampires@lemmy.zip
- !weirdcore@sopuli.xyz
- !witchcraft@lemmy.ml *
- !witchesvspatriarchy@lemmy.ca
HISTORY
- !ancientcoins@lemmy.world
- !ancienthistory@slrpnk.net
- !artp@sh.itjust.works *
- !askhistorians@lemmy.world *
- !historyartifacts@lemmy.world
- !historicalpropaganda@lemmy.blahaj.zone
- !history@lemmy.world
- !historyporn@lemmy.world
- !historyruins@lemmy.world
- !historyillustrations@lemmy.world
- !vintageads@sh.itjust.works *
INFORMATION / KNOWLEDGE
- !buyitforlife@slrpnk.net
- !catastrophicfailure@lemmy.world *
- !creepywikipedia@lemmy.world
- !dataisbeautiful@lemmy.ml
- !documentaries@lemmy.cafe
- !esa@feddit.nl
- !explainlikeimfive@lemmy.world
- !goodnewseveryone@sh.itjust.works
- !humanities@beehaw.org
- !interestingasfuck@lemmy.world
- !math@lemmy.world
- !mealtimevideos@lemmy.cafe
- !recommendations@lemmy.world
- !solarpunk@slrpnk.net
- !thingscutinhalf@lemmy.world
- !til@lemmy.world
- !vexillology@lemmy.world
- !whatisthisthing@lemmy.world
- !wikipedia@lemmy.world
- !youshouldknow@lemmy.world
OTHER
- !andfinally@feddit.uk
- !academiagonewild@lemmy.world *
- !internetisbeautiful@lemm.ee
- !mildlyinteresting@lemmy.world
- !nowhereelsetoshare@sh.itjust.works
- !pbsod@lemmy.ohaa.xyz
- !publictransport@slrpnk.net
- !stick@sh.itjust.works *
- !sumo@lemmy.world *
- !shortstories@literature.cafe
- !treehuggers@slrpnk.net
- !the_heinlein_society@lemmy.world
- !woahdude@lemmy.world *
- !youcantparktheremate@feddit.uk *
FEDIVERSE
- !fedibridge@lemmy.dbzer0.com
- !fedigrow@lemm.ee
- !fediverse@lemmy.world
- !fediverse@piefed.social
- !fediverselore@lemmy.ca
- !lemmyapps@lemmy.world
- !comm_revival@lemmy.dbzer0.com *
- !yepowertrippinbastards@lemmy.dbzer0.com
FINDING NEW/GOOD COMMUNITIES ON LEMMY
- !communities@ponder.cat
- !communitypromo@lemmy.ca
- !findacommunity@lemmy.ml
- !index@lemmy.world *
- !lemmy411@lemmy.ca
- !newcommunities@discuss.online
- !newcommunities@lemmy.world
- !trendingcommunities@feddit.nl
- !wowthislemmyexists@lemmy.ca *
:::
::: spoiler click here for a list of (mostly non-political) meme communities
MEMES, SOCIAL MEDIA REPOSTS, AND HUMOR
Most of these are currently active. (except for the ones with a * which were less active last I checked). Sometimes these include politics but that's not the sole focus.
- !onehundredninetysix@lemmy.blahaj.zone
- !196@pawb.social
- !19684@lemmy.blahaj.zone
- !2meirl4meirl@lemmy.world
- !4chan@lemmy.world
- !4chanFails@feddit.nl
- !absoluteunit@sh.itjust.works
- !ancientinternet@lemmy.world
- !animemes@ani.social
- !antimeme@lemmy.world *
- !atbge@lemmy.world
- !antiquememesroadshow@lemmy.world
- !are_the_straights_ok@lemmy.blahaj.zone *
- !atheistmemes@lemmy.world
- !badrealestate@feddit.uk *
- !bonehurtingjuice@lemmy.ca
- !coaxedintoasnafu@lemmy.dbzer0.com *
- !comedyheaven@lemmy.world
- !Couplememes@sh.itjust.works
- !crumbgrabber@lemm.ee *
- !curatedtumblr@sh.itjust.works
- !dankchristianmemes@lemmy.blahaj.zone
- !dankmemes@lemmy.world
- !ehbuddyhoser@sh.itjust.works
- !ExplainTheJoke@spaffel.social
- !fbstolencontent@lemmy.ca *
- !fediashitpost@fedia.io
- !fedimemes@feddit.uk
- !femcelmemes@lemmy.blahaj.zone
- !funny@sh.itjust.works
- !funny@lemmy.world
- !furry_irl@pawb.social
- !garfieldism@lemmy.blahaj.zone *
- !gifconversation@lemm.ee
- !greentext@sh.itjust.works
- !gunmemes@lemmy.world *
- !high_quality_gifs@lemmy.sdf.org
- !hq_memes@lemmy.world
- !imgoingtohellforthis@lemm.ee
- !internetfuneral@lemmy.world
- !humor@beehaw.org *
- !iamverybadass@lemmy.ca *
- !justpost@lemmy.world
- !kemper_loves_you@lemmy.dbzer0.com *
- !lemmybewholesome@lemmy.world
- !lemmyshitpost@lemmy.world
- !linkedinlunatics@sh.itjust.works *
- !lotrmemes@lemmy.dbzer0.com
- !loweffortmemes@crazypeople.online
- !mathmemes@lemmy.blahaj.zone
- !me_irl@lemmy.world
- !memes@jlai.lu
- !memes@lemmy.world
- !memes@slrpnk.net
- !memes@sopuli.xyz
- !memes@lemmy.ml
- !metalmemes@lemmy.world
- !microblogmemes@lemmy.world
- !MoldyMemes@spaffel.social
- !NiceMemes@sopuli.xyz *
- !nocontextpics@lemmy.world
- !nominativedeterminism@feddit.uk *
- !nonsense@moist.catsweat.com *
- !okmatewanker@feddit.uk
- !potatoism@lemmy.world *
- !pokemonmemes@lemmy.world *
- !programmer_humor@programming.dev
- !reactionmemes@lemmy.dbzer0.com *
- !reallyshittycopper@lemmy.world *
- !relationshipmemes@lemmyis.fun
- !roughromanmemes@lemmy.world
- !rpgmemes@ttrpg.network
- !RichardScarryPosting@sh.itjust.works
- !shitty_million_dollar_ideas@lemmy.dbzer0.com
- !SighFi@quokk.au
- !science_memes@mander.xyz
- !shera@lemmy.blahaj.zone
- !shirtsthatgohard@lemmy.world
- !shitposting@lemmy.ml
- !shittylifeprotips@lemmy.world
- !shittytattoos@lemmy.ca
- !spaceflightmemes@sh.itjust.works
- !surrealmemes@sh.itjust.works *
- !tech_memes@lemmy.world *
- !tantes_linksqueer@feddit.org
- !the_pack@lemmy.world
- !TwoGoobers@lemm.ee
- !wholesomegreentext@sh.itjust.works *
- !wholesomememes@lemmy.world *
- !wizards@lemmy.world
- !witchymemes@lemmy.world
- !wlw_memes@lemmy.world
- !yurimemes@lemmy.blahaj.zone
- !youtubeclassics@sh.itjust.works
:::
(luckily I was posting this to newcomers whenever I had the chance...)
-
Alt of ProdigalFrog@slrpnk.net chiming in, thanks for giving us the sitrep, poVoq!
This is an unusually unlucky set of circumstances, but for the Solarpunks who find their way here, let's try to make the best of it. If you don't already have an alternative account somewhere, you may want to create one to use until we're back up and running. lemmy.dbzer0.com, sopuli.xyz, or lemmy.cafe are some solid choices to consider. After that, you may want to favorite poVoq's post here so you can check back on any news.
Even without our home base, we can still share the solarpunky stuff we find and create around the threadiverse (though as Tofu mentioned, maybe this could be a time to take a breather to be refreshed when we come back
).Besides, if the past is any indication, time will fly by and we'll be back before you know it!
Lol love the new username, fitting!
