3751-3800 of 10000 results (13ms)
2011-06-24 §
19:20 <Ryan_Lane> depooled srv301 [production]
12:37 <Tim-away> finished fixing ExtensionDistributor, re-enabled cron job and xinetd [production]
07:37 <Tim> XO appears to be back up, after ~3 hours of downtime [production]
07:30 <Tim> fixing ExtensionDistributor snapshot permissions, was owned by mark instead of extdist [production]
06:21 <jeluf> synchronized php-1.17/wmf-config/InitialiseSettings.php '29168 - Define namespace aliases for Author, Page, Index on all Wikisources' [production]
06:10 <Tim> configured csw5-pmtpa to send its syslogs to 10.0.5.8 instead of 10.0.0.1 [production]
00:19 <tstarling> synchronized php-1.17/includes/db/DatabaseMysql.php 'reapplied r90644, it was fixing an actual problem' [production]
2011-06-23 §
22:36 <pdhanda> synchronized php-1.17/wmf-config/InitialiseSettings.php 'Bug 14407. Added Login icon for incubatorwiki' [production]
22:33 <pdhanda> synchronized images/sul/incubatorwiki.png 'Bug 14407. Login icon for incubatorwiki' [production]
22:18 <pdhanda> synchronized php-1.17/wmf-config/CommonSettings.php 'Bug 14407. Added incubatorwiki to CentralAuthAutoLoginWikis' [production]
21:30 <catrope> synchronized php-1.17/maintenance/update.php 'Sync out update.php too, just because Ariel is being paranoid about it :D' [production]
21:27 <^demon> svn up'd /h/w/c/p/m/update.php for nice warning about not running update.php on the cluster [production]
21:06 <pdhanda> ran sync-common-all 'Updating to r90676 of extensions/UploadWizard' [production]
21:03 <awjrichards> synchronized php-1.17/wmf-config/CommonSettings.php 'Enabling wgNoticeInfrastructure for testwiki per request by Kaldari' [production]
20:36 <pdhanda> updated to r90667 of extensions/Narayam [production]
20:33 <pdhanda> ran sync-common-all [production]
18:38 <JeLuF> #21237 - removed brion's email address from apache config [production]
18:10 <jeluf> synchronized php-1.17/wmf-config/InitialiseSettings.php '28909 - Change path to Low Saxon Wiktionary logo' [production]
18:04 <jeluf> synchronized php-1.17/wmf-config/InitialiseSettings.php '29391 - Enable e-mail notification for watchlists on br.wikimedia' [production]
18:03 <jeluf> synchronized php-1.17/wmf-config/InitialiseSettings.php '29348 - Email notifications for stewardwiki' [production]
18:00 <jeluf> synchronized php-1.17/wmf-config/InitialiseSettings.php '29360 - Define $wgUploadMissingFileUrl for ptwikibooks' [production]
17:02 <binasher> sync-file r90664 includes/db/LoadMonitor.php [production]
17:01 <root> synchronized php-1.17/includes/db/LoadMonitor.php 'max threads logging' [production]
16:47 <tstarling> synchronized php-1.17/wmf-config/db.php 'attempting to set max threads on testwiki only' [production]
16:32 <RoanKattouw> Site came back up instantly after Tim disabled max_threads [production]
16:28 <tstarling> synchronized php-1.17/wmf-config/db.php 'disabled max threads' [production]
16:28 <catrope> synchronized php-1.17/includes/db/LoadBalancer.php 'Make the "LB failure with no last connection" DB error log message more verbose' [production]
16:24 <binasher> upping max threads in db.php [production]
16:24 <root> synchronized php-1.17/wmf-config/db.php 'upping max threads' [production]
16:17 <laner> synchronized php-1.17/includes/db/DatabaseMysql.php 'Roll back Tim's getLag() change from 03:18 today (r90644)' [production]
15:58 <Ryan_Lane> restarting pybal on lvs3 [production]
15:52 <catrope> synchronized php-1.17/includes/db/Database.php 'Roll back Tim's getLag() change from 03:18 today (r90644)' [production]
15:50 <RoanKattouw> Going to roll this back: 03:18 logmsgbot: tstarling synchronized php-1.17/includes/db/DatabaseMysql.php 'Database::getLag() fix from r90644' [production]
15:46 <RoanKattouw> Lots of "LB failure with no last connection" errors in dberror.log, suddenly started at 10:53 UTC [production]
15:43 <RoanKattouw> Torrus broken, reporting lots of NaNs [production]
15:40 <RoanKattouw> Strike that, site not completely down. Intermittent service reported in US/Canada [production]
15:39 <RoanKattouw> AMS squids all down in Nagios. Load is approximately halved on app servers. TPA squids look like they should be fine, but the site is reported to be down worldwide [production]
15:37 <RoanKattouw> Mark not near a computer, texting Ryan instead [production]
15:36 <RoanKattouw> All wikis seem to be down worldwide. Have texted mark [production]
09:18 <jeluf> synchronized php-1.17/wmf-config/InitialiseSettings.php '29215 - Add import source for dearbcomwiki' [production]
09:14 <jeluf> synchronized php-1.17/wmf-config/InitialiseSettings.php '29380 - Change logo of Kannada wiktionary (kn.wiktionary.org)' [production]
08:48 <jeluf> synchronized php-1.17/wmf-config/InitialiseSettings.php '29515 - install Narayam in all Sanskrit wiki projects' [production]
08:42 <jeluf> synchronized php-1.17/wmf-config/InitialiseSettings.php '29474 - Search Index and Author namespaces by default on all Wikisources which don't have a different configuration' [production]
08:27 <jeluf> synchronized php-1.17/wmf-config/InitialiseSettings.php '29172 - Add Page, Author and Index namespaces to $wgContentNamespaces on all Wikisources' [production]
07:53 <JeLuF> fixed broken ganglia installation on project2 (reported by puppetmon) [production]
07:51 <JeLuF> added new class mobile::disabled in puppet that has apache disabled, assigned mobile1 to it. [production]
07:19 <tstarling> synchronized php-1.17/extensions/SubPageList3/SubPageList3.php 'r90654' [production]
07:12 <tstarling> synchronized php-1.17/wmf-config/CommonSettings.php 'enabling SubPageList3 on testwiki' [production]
07:11 <tstarling> synchronized php-1.17/wmf-config/InitialiseSettings.php [production]
07:07 <root> synchronizing Wikimedia installation... Revision: 90493: [production]