Backlog of stats....

This forum is for discussions about the Motherboards.org Folding team. What is folding? Venture on in for a look.

Moderator: The Mod Squad

Backlog of stats....

Postby whiteboy » Sun Apr 27, 2008 3:46 am

In case anybody was also wondering about stats and why they might be missing credit. see my previous post

taken from the folding forums...
We've been experiencing a great deal of stats instability, due to the new advmethods projects announced recently. The problem is that these WU's are just too short for the stats system to handle, flooding the system with 10x more WU's than it's used to handling. That means that instead of completing an update in 1 hour, it now takes 10hours, which is a disaster.

We're working to correct this, but it will likely take a little time to get through the backlog that's been generated.

As I've posted below, we have a backlog of stats to input into our db for donors to access their scores. To speed this process, we've made some temporary changes. We've disabled stats updates from our web site. We've also limited our updates on the external stats pages for teams and donors to once every 12 hours at 6am and 6pm PST. We're trying to streamline the process so that we can get the backlog through and back to business as normal.

This is just temporary, but will be a big help. Once the backlog is through, the points will be up and hopefully all will be back to normal. We expect this may take as long as 2 days, and will give updates along the way.

The last stats update just finished (taking about 6 hours). The next one should be a lot faster since there's less stats built up, and the next one faster yet. We'll keep the outside access to the db down until we're back to every 2 hours though. It looks like that will be tomorrow morning (Sunday morning PST).

We've got the stats back to their regular 2 hour updates, although we're keeping the 3rd party stats still to every 12 hours until Sunday morning PST. We've cleaned up some aspects of how the internal stats scripts work and see a way to speed up the stats input significantly (perhaps 4x), but we'll leave that until this mess has blown over before we start trying to make any further improvements (to avoid introducing any new errors at a sensitive time).
Amd Phenom 9500 Quad on a ECS a770m mobo
4 gigs ram
ATI Radeon x1950pro w/1 gig ram video card
2 250 gig samsung sata HDs
2 lite on sata dvd burners
all in a raidmax blue neon case running Vista64
total cost under $600 built march 2008
whiteboy
Anti-Static Strap
Anti-Static Strap
 
Posts: 439
Joined: Wed Jan 15, 2003 8:36 pm
Location: Lehigh Acres, Fl

Postby Karlsweldt » Sun Apr 27, 2008 1:40 pm

Nice job with the research!
Those WU servers may take in several hundred projects each, per hour.. and keeping track autonomously sometimes won't work.
Not sure, but I believe all the servers are at Stanford, in California, U.S.A.
Here is a listing of server status if interested.
F@H.. to solve mankind's maladies.. in our lifetimes!
Karlsweldt
Mobo-fu Master
Mobo-fu Master
 
Posts: 20659
Joined: Wed Nov 12, 2003 11:57 am
Location: 07438

Postby whiteboy » Mon Apr 28, 2008 3:31 am

update::

Stats back to regular updates (everything)
We put the user and team file updates back to every 3 hours (used by 3rd party stats), so we're now back to regular behavior. It looks like we should be ok from here on out.

There are two upshots of this mess. First, we've developed some new emergency procedures to deal with such back logs better in the future. We also have plans for how to refactor the stats input code to potentially speed up the process by 5x (at least 2x). That should help in general (perhaps lettings us go back to hourly updates).
Amd Phenom 9500 Quad on a ECS a770m mobo
4 gigs ram
ATI Radeon x1950pro w/1 gig ram video card
2 250 gig samsung sata HDs
2 lite on sata dvd burners
all in a raidmax blue neon case running Vista64
total cost under $600 built march 2008
whiteboy
Anti-Static Strap
Anti-Static Strap
 
Posts: 439
Joined: Wed Jan 15, 2003 8:36 pm
Location: Lehigh Acres, Fl


Return to Motherboards.org Folding Team

Who is online

Users browsing this forum: No registered users and 1 guest