We are back............ I hope

Stuart Ablett

Member
Messages
15,917
Location
Tokyo Japan
From Marty the Admin guy............

Folks,

As many/most of you already know, the forum has been down since some time
last evening (after midnight EST).

I've been in contact with Bluehost (our provider) on and off all morning.
What I learned is that last night, during a routine nightly scan of the file
system on the actual server (machine) we run on, it was discovered that the
file system was corrupt.

A process was started shortly after midnight to correct the problem. This
process 'normally' runs for about 15 minutes, corrects the problem, and
we're back up and running. Well, the process ran for hours on end and did
not correct the problem.

Bluehost was running diagnostics when I first contacted them early this
morning. They discovered that a rebuild of the entire file system was
required, and they were unable to give me an estimate on the time it would
require.

Unfortunately, given the odd nature of the problem, Bluehost didn't expect
us to be down for very long, so they didn't move us (and the hundreds of
other sites we share the server with) to another server. Had they attempted
to move us to an alternate server, we would have been running with a backup
of the filesystem from the previous day. That would have created a host of
problems all by itself!

I apologize for this down time. In the end, I still maintain that we're
doing pretty good. In the time we've been up, this is probably one of the
longest down times we've experienced, and it was the hosts fault.

I'm off to take an unscheduled/early backup of the entire forum...

Sorry again...
- Marty -

Hope it all stays up now! :thumb:
 
Last edited by a moderator:
No need to apologize, though I did kind of miss everyone for those few hours. Glad your back up and running. Don't forget to thank BlueHost either for their hard work. I know they are to blame, but its better to give them a nod for their hard work rather then complain and get put on the back burner the next time this happens.
 
No reason to apologize - especially since I have no idea what you just said.:rolleyes: But thanks for getting things up and keeping tabs on the whole process.

Wes
 
I was just cleaning up the server room a bit...needed a spare outlet so I could plug in the vacuum cleaner, but didn't realize until it was too late that the black cord is the server, the gray cord is the pencil sharpener. Shoulda pulled the gray one. :doh: :rofl:
 
Well, we WERE back up...then I got involved.

I personally apologize for the slew of CPU exceeded messages I'm sure everyone has been getting for the last little while.

It was for a good cause though. I was in the process of BACKING UP THE ENTIRE FORUM. Boy, do we have a bunch of bytes! The filesystem is over 1.3 GIGABYTES, and the Database is almost 20 MEGABYTES.

(For all you non-geeks, that's a BUNCH OF STUFF! :rofl: )

The backup took me over four hours, and ran the CPU on the server into overload mode, which slowed the process down even more.

But, now I can rest knowing that we have a backup of EVERYTHING as of right now. Even if there was a catastrophic hardware failure at Bluehost, we'd only loose stuff from now (til the next backup...:doh: ) Oh, and I have a copy of EVERYTHING on my personal PC, as well as a copy on a remote harddrive...just in case. Took a while longer...but it's cheap insurance. :thumb:

Seriously, I do backups weekly, but I ALWAYS do them very late at night and let them run through the night. There are so few people on the forum then that we usually don't have CPU quota problems. Given our current situation, recovering from a problem, with lots of people logged on to get their "fix", I guess I pushed things too hard.

Sorry...but I'm glad I did it! :eek:
- Marty -
 
I was just cleaning up the server room a bit...needed a spare outlet so I could plug in the vacuum cleaner, but didn't realize until it was too late that the black cord is the server, the gray cord is the pencil sharpener. Shoulda pulled the gray one. :doh: :rofl:

good`n vaughn!
 
Seriously, I do backups weekly, but I ALWAYS do them very late at night and let them run through the night. There are so few people on the forum then


Night: Noun.
1: the time after sunset and before sunrise while it is dark outside
2: Early afternoon / Broad daylight in Japan, when Stu is bored out of his gourd in the L shop but unable to get on the forum thanks to Marty... :doh: :rofl: :D
 
Night: Noun.
1: the time after sunset and before sunrise while it is dark outside
2: Early afternoon / Broad daylight in Japan, when Stu is bored out of his gourd in the L shop but unable to get on the forum thanks to Marty... :doh: :rofl: :D


Well Art,

Let me know what time would be convenient for EVERYONE, and I'll see if I can make that the time when I take full forum backups...:dunno:

Hey, it's momentary, and hopefully Stu has a customer or two when the CPU starts glowing...:rolleyes:

- Marty -
 
Thanks for the hard work Marty. Some may not realize how important a full and incremental backups are, until it is too late. Hope all is well and you are making progress in the "studio".
 
Night: Noun.
1: the time after sunset and before sunrise while it is dark outside
2: Early afternoon / Broad daylight in Japan, when Stu is bored out of his gourd in the L shop but unable to get on the forum thanks to Marty... :doh: :rofl: :D

Uh.. I see Art's humor, anyway...:thumb:

Marty must have chipped a shaper bit...:eek:

:rofl: :rofl: :rofl:
 
Let me know what time would be convenient for EVERYONE,

:huh: :huh: :huh:

I dunno if that is possible, given that Stu seems to get by on about 4hrs of sleep a night. :eek:

No worries, Greg, I could see that Marty was just teasing me back.
In all seriousness, my hat is off to Marty and the whole admin team for this and many other acts of service.

...art
 
Top