651-700 of 10000 results (34ms)
2016-03-09 §
10:45 <joal> Deploying refinery from Tin [analytics]
10:39 <elukey> puppet disabled on rdb1004 (plus Redis instances) as precautionary step for master reimage - rdb1003 [production]
10:35 <jynus> setting up master-master replication on tools (labsdb100[45]) [production]
10:29 <hashar> Nodepool: created new image and refreshing snapshot in attempt to get Xvfb running T129320 T128090 [releng]
10:15 <joal> Killing oozie jobs before deploy and restart with new error email configuration [analytics]
09:32 <moritzm> rebooting iron for kernel update [production]
09:26 <elukey> puppet disabled on rdb1003 for debian re-image. Redis servers going to be stopped too as pre-step for backup. [production]
08:32 <oblivian@tin> Synchronized wmf-config/jobqueue-eqiad.php: re-routing writes from rdb1003 to rdb1005 for reimaging (duration: 00m 36s) [production]
08:29 <jynus@tin> Synchronized wmf-config/db-codfw.php: Repool db2035 (duration: 00m 39s) [production]
07:49 <_joe_> uploading hhvm 3.12, extensions to reprepro [production]
06:46 <joal> Deploying refinery-source v0.0.27 to archiva [analytics]
06:26 <mutante> bast2001 - still installing in snail mode - please feel free to check if it's done, and if so re-add to puppet so users get created.thx [production]
03:16 <l10nupdate@tin> ResourceLoader cache refresh completed at Wed Mar 9 03:16:21 UTC 2016 (duration 8m 2s) [production]
03:08 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.16) (duration: 17m 51s) [production]
02:32 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.15) (duration: 13m 28s) [production]
02:13 <ori@tin> Synchronized php-1.27.0-wmf.15/includes/diff/DairikiDiff.php: I4d4b8f81c: Dont quote assert expressions in DairikiDiff (duration: 00m 27s) [production]
02:12 <ori@tin> Synchronized php-1.27.0-wmf.16/includes/diff/DairikiDiff.php: I4d4b8f81c: Dont quote assert expressions in DairikiDiff (duration: 00m 31s) [production]
01:40 <csteipp> redeploy patch for T129120 [production]
01:39 <csteipp@tin> Synchronized php-1.27.0-wmf.16/includes/api/ApiQueryInfo.php: (no message) (duration: 00m 33s) [production]
00:33 <maxsem@tin> Synchronized php-1.27.0-wmf.15/extensions/Echo/: SWAT (duration: 00m 32s) [production]
00:32 <maxsem@tin> Synchronized php-1.27.0-wmf.15/extensions/MobileFrontend/: SWAT (duration: 00m 32s) [production]
00:30 <maxsem@tin> Synchronized php-1.27.0-wmf.16/extensions/MobileFrontend/: SWAT (duration: 00m 29s) [production]
00:29 <bd808> Checking to see if bot is really listening in #wikimedia-labs [tools.stashbot]
00:29 <maxsem@tin> Synchronized php-1.27.0-wmf.16/extensions/Echo/: SWAT (duration: 00m 38s) [production]
00:25 <maxsem@tin> Synchronized wmf-config/: https://gerrit.wikimedia.org/r/275378 - comment only (duration: 00m 27s) [production]
00:16 <maxsem@tin> Synchronized wmf-config/filebackend-production.php: https://gerrit.wikimedia.org/r/272922 (duration: 00m 27s) [production]
00:13 <maxsem@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/275858/ (duration: 00m 28s) [production]
00:09 <maxsem@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/275275 (duration: 00m 29s) [production]
00:06 <bd808> Added Greg Grossmeier as co-maintainer [tools.stashbot]
00:05 <bd808> Restarted bot to clear busted phabricator session [tools.stashbot]
2016-03-08 §
23:42 <legoktm> running CentralAuth's checkLocalUser.php --verbose=1 --delete=1 on deployment-tin for T115198 [releng]
23:20 <mutante> bast2001 - install issues - extending downtime, bbiaw [production]
23:20 <twentyafterfour@tin> rebuilt wikiversions.php and synchronized wikiversions files: group0 to 1.27.0-wmf.16 [production]
23:11 <mutante> neon: re-enable puppet, start icinga-wm [production]
23:08 <mafk> chmod 770 to a folder [tools.stewardbots]
23:03 <twentyafterfour@tin> Synchronized w/static/: (no message) (duration: 00m 32s) [production]
23:01 <mafk> moved some files from old folders, miscellaneous fixes [tools.stewardbots]
22:53 <mutante> bast2001 - powercycle, reinstall [production]
22:19 <twentyafterfour@tin> Finished scap: testwiki to php-1.27.0-wmf.16 and rebuild l10n cache (duration: 26m 06s) [production]
22:04 <ori> Canary application servers (mw1017-mw1025) and canary API application servers (mw1114-mw1119) upgraded to HHVM 3.12.1 [production]
22:04 <jynus> recreating slave watchdog events on all servers [production]
21:53 <twentyafterfour@tin> Started scap: testwiki to php-1.27.0-wmf.16 and rebuild l10n cache [production]
21:33 <hashar> Nodepool Image ci-jessie-wikimedia-1457472606 in wmflabs-eqiad is ready [releng]
21:21 <mafk> Merged https://gerrit.wikimedia.org/r/275886 [tools.stewardbots]
21:02 <jynus@tin> Synchronized wmf-config/db-codfw.php: Repool db2040 (duration: 00m 34s) [production]
21:01 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Repool db1066 (duration: 00m 33s) [production]
20:49 <ori> Updated mw1200 to HHVM 3.12.1; repooling [production]
20:45 <ori> Depooled mw1200 for HHVM update [production]
20:40 <jynus> killing long running queries on db1066 [production]
20:39 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Depool db1066 (duration: 01m 49s) [production]