Tigris being restarted

Tigris.Org is being restarted to fix an unplanned outage. Should be back on-line within about 15 minutes.

Update: well, it took just over three hours, not 15 minutes, but we’ve stabilized again and are back in service.

The root cause was a conflux of several problems (a configuration setting, some ill-advisadly permissible user-entered text, and a failure to filter stuff out) that  have been or will all be fixed.

Brief mail problems on Tigris, now fixed

For a few minutes today, around 1:15 PM PDT, users may have experienced some mail bouncing from Tigris.Org. The problem, a failed mail hum / spam blocker, was fixed by about 1:30. Service should now be fully restored.

Tigris Upgrade: Timeline and chance to help

The upgrade of Tigris to CollabNet Enterprise Edition 5.3 is on track and coming soon. The current schedule is:

Community testing                             10d         Tue 1/26/10      Mon 2/8/10

Production upgrade of tigris to 5.3.0          1d         Fri 2/12/10      Fri 2/12/10

So next Tuesday, we’ll be looking for your help: we’ll have a “staging” site set up, a copy of the Tigris site but with the new software. We want your help to test it out, comment on the new features, and find the new bugs.

To participate, subscribe to the “staging” list at Tigris.Org (send email to staging-subscribe@tigris.org). That’s where you’ll see the announcement, and connection details. That’s also where discussions about the upgrade should happen. (Note that this list is on the real Tigris.Org, even though it’s about the test site. That’s so we can continue conversations even if we manage to crash the test site.)

So, sign up now, and watch your in-box!

Update (2/2/10):

Well, is it January 26, yet?

Some problems in upgrading a large site like Tigris were discovered and corrected, but that means we have to start the set-up and staging of the site again. New schedule:

Community testing: 10 days, starting Feb 22 (through March 5)

Actual upgrade: Second week in March

January Maintenance: An Update

Here are additional details on plans for upgrading the Tigris site:

• Add additional memory to improve performance: 1/21. Complete.
• Set up a “staging server” (or “beta,” if you like) for Project Owners and users to test: 1/26 – 2/5
• Upgrade Tigris.Org to CEE 5.3.0 patch 2 ‘Indus’

Not that “staging server” item, there. We’ll be asking all Tigris users to beat up the staging server for about ten days, to help us ensure a smooth transition. Stay tuned here, in Twitter (@tigrisdotorg), or on the “announce” list at tigris.org.

January maintenance activities on Tigris

Several maintenance activities are planned for Tigris during January:

• On January 6, at 2:00PM PST, the site will be down for about one hour, to add additional memory, which should improve performance
• Within a week or so, a staging site will be established in preparation for a software upgrade. Project Owners and interested others will be asked to verify that their staged projects look and behave properly. Staging does not involve any downtime for Tigris itself.
• The actual upgrade of Tigris will happen by the end of the month. This will require some downtime

Follow this blog or the announcements list at Tigris for continuing details.

“Service Temporarily Unavailable” (status 503) from SVN

Subversion service on Tigris is currently running at reduced capacity, and may return an error 503 “Service Temporarily Unavailable.”  This is because we have throttled Subversion access, per project, a bit while we work on a memory-leakage bug just discovered.

If you encounter this problem, please wait a minute or so and retry your operation. If that’s unsuccessful, contact us, and we’ll look into raising the threshold for your project. You can do that via comments on this blog, through the Twitter account @tigrisdotorg, or via email to the feedback address at tigris.org.

The fix for the underlying problem is known, and already proven in several other production sites. We’re working on getting it installed onto Tigris as quickly as possible.

Coming upgrades on Tigris

We’ll be upgrading Tigris to the latest platform version, CollabNet Enterprise Edition 5.3 ‘Indus’ next month. This upgrade fixes the last, lingering glitches of the Discussion Services upgrades. It also includes some notable performance boosts, both in web use and in Subversion access, and it relieves some barriers to growth and scalability. All in all, it’s something we’ve been wanting for quite some time.

As both preparation and a sort of down-payment, there will be a brief downtime this evening, at 18:00 PST. This down time should be about 20 minutes.