|
2016-01-26
§
|
| 14:14 |
<akosiaris> |
migrate alsafi,mx2001 back from ganeti2004 to fix a network misconfiguration |
[production] |
| 13:32 |
<moritzm> |
rebooted nescio/maerlant for kernel update |
[production] |
| 13:14 |
<jynus@mira> |
Synchronized wmf-config/db-eqiad.php: Depool pc1002 for maintenance (clone to pc1005) (duration: 01m 39s) |
[production] |
| 12:39 |
<akosiaris> |
rolling reboot of ganeti200{1,2,3,4,5,6}.codfw.wmnet for kernel upgrade |
[production] |
| 12:10 |
<moritzm> |
rebooting mx2001/mx1001 (with a delay in between) for kernel update |
[production] |
| 11:50 |
<moritzm> |
rebooting etherpad1001 for kernel update |
[production] |
| 11:46 |
<moritzm> |
rebooting bromine for kernel update |
[production] |
| 10:50 |
<ema> |
Starting migration of mobile traffic to text cluster in esams https://phabricator.wikimedia.org/T109286 |
[production] |
| 09:30 |
<hashar> |
restarting Jenkins to upgrade the gearman plugin with https://review.openstack.org/#/c/271543/ |
[production] |
| 09:28 |
<_joe_> |
finishing reboots of appservers in eqiad |
[production] |
| 04:27 |
<legoktm> |
restarted resetGlobalUserTokens.php after it lost mysql connection again |
[production] |
| 02:30 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Tue Jan 26 02:30:58 UTC 2016 (duration 7m 0s) |
[production] |
| 02:23 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.10) (duration: 09m 36s) |
[production] |
| 01:45 |
<krenair@mira> |
Synchronized wmf-config/wikitech.php: https://gerrit.wikimedia.org/r/#/c/266453/ (duration: 01m 27s) |
[production] |
| 00:45 |
<mobrovac> |
mobileapps deploying c2318b6 |
[production] |
| 00:40 |
<ebernhardson@mira> |
Synchronized wmf-config/CommonSettings.php: (no message) (duration: 01m 25s) |
[production] |
| 00:37 |
<ebernhardson@mira> |
Synchronized wmf-config/InitialiseSettings.php: SWAT bd808 (duration: 01m 34s) |
[production] |
| 00:32 |
<ebernhardson@mira> |
Synchronized portals/: SWAT jgirault (duration: 01m 28s) |
[production] |
| 00:29 |
<ebernhardson@mira> |
Synchronized wmf-config/InitialiseSettings.php: SWAT ebernhardson (duration: 01m 26s) |
[production] |
| 00:27 |
<ebernhardson@mira> |
Synchronized wmf-config/CirrusSearch-common.php: SWAT ebernhardson (duration: 01m 26s) |
[production] |
| 00:25 |
<ebernhardson@mira> |
Synchronized wmf-config/CommonSettings.php: SWAT ebernhardson (duration: 01m 27s) |
[production] |
| 00:15 |
<ebernhardson@mira> |
Synchronized wmf-config/CommonSettings.php: SWAT AaronSchulz (duration: 01m 26s) |
[production] |
| 00:13 |
<ebernhardson@mira> |
Synchronized wmf-config/filebackend-production.php: SWAT AaronSchulz (duration: 01m 26s) |
[production] |
| 00:10 |
<ebernhardson@mira> |
Synchronized wmf-config/CommonSettings.php: SWAT James_F (duration: 01m 26s) |
[production] |
| 00:08 |
<ebernhardson@mira> |
Synchronized wmf-config/InitialiseSettings.php: SWAT James_F (duration: 01m 35s) |
[production] |
|
2016-01-25
§
|
| 23:14 |
<legoktm@mira> |
Synchronized php-1.27.0-wmf.10/includes/parser/: live hacks, now committed (duration: 01m 27s) |
[production] |
| 23:07 |
<legoktm@mira> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/266410/ (duration: 01m 35s) |
[production] |
| 22:52 |
<ori@mira> |
Synchronized php-1.27.0-wmf.10/includes/parser/ParserOutput.php: Fix-up for ParserOutput.php@263 debug logging (duration: 01m 27s) |
[production] |
| 22:30 |
<legoktm@mira> |
Synchronized php-1.27.0-wmf.10/includes/parser/: https://gerrit.wikimedia.org/r/#/c/266401/ + https://gerrit.wikimedia.org/r/#/c/266406/ + live hacks (duration: 01m 28s) |
[production] |
| 22:28 |
<legoktm@mira> |
Synchronized php-1.27.0-wmf.10/includes/content/WikitextContent.php: https://gerrit.wikimedia.org/r/#/c/266401/ (duration: 01m 29s) |
[production] |
| 21:53 |
<hoo@mira> |
Synchronized wmf-config/Wikibase-production.php: Disable (not yet deployed) commons category sidebar link overwrite in production (duration: 01m 28s) |
[production] |
| 21:47 |
<mutante> |
nitrogen - shutdown -h now .... |
[production] |
| 21:45 |
<mutante> |
alsafi - was reported down in icinga , is ganeti VM - fixed by just logging in as if it went to hibernate |
[production] |
| 21:37 |
<mdholloway> |
mobileapps deployed 9252a22 |
[production] |
| 21:30 |
<mutante> |
nitrogen - stop puppet, stop salt, remove from stored configs / icinga |
[production] |
| 20:19 |
<hoo@mira> |
Synchronized wmf-config/Wikibase-labs.php: (no message) (duration: 01m 28s) |
[production] |
| 20:14 |
<chasemp> |
bump labstore nfs threads to 288 from 244 |
[production] |
| 19:32 |
<paravoid> |
eqiad: removing static routes for 6to4/Teredo to nitrogen (decommissioning our own relays) |
[production] |
| 19:10 |
<bd808> |
Live hacking on mw1017 to debug 1.27.0-wmf.11 issues. All wikis there currently set to use 1.27.0-wmf.11. |
[production] |
| 19:05 |
<chasemp> |
labstore1001 temp change to CFQ scheduler on 01/22/2015 |
[production] |
| 19:04 |
<chasemp> |
the nfsd thread change is on labstore1001 |
[production] |
| 19:04 |
<chasemp> |
nfsd has 224 threads atm and was bumped up over the weekend |
[production] |
| 18:58 |
<ori> |
removed unused wikiversions.cdb on mira and tin |
[production] |
| 18:28 |
<jynus> |
retroactively logging the depool of mw1217, mw1178 and mw1257 3 hours ago (Jan 25 15:45:26) |
[production] |
| 16:49 |
<ema> |
Finished migration of mobile traffic to text cluster in ulsfo https://phabricator.wikimedia.org/T109286 |
[production] |
| 16:38 |
<jynus@mira> |
Synchronized wmf-config/db-eqiad.php: Preparing ips for new parsercache deployments (third try) (duration: 01m 35s) |
[production] |
| 16:26 |
<jynus@mira> |
Synchronized wmf-config/db-eqiad.php: Preparing ips for new parsercache deployments (second try after running puppet) (duration: 03m 23s) |
[production] |
| 16:25 |
<_joe_> |
restarting salt-minion on all deployment targets |
[production] |
| 16:24 |
<_joe_> |
running salt deploy.fixurl on all deployment targets |
[production] |
| 16:09 |
<jynus@mira> |
Synchronized wmf-config/db-eqiad.php: Preparing ips for new parsercache deployments (duration: 03m 32s) |
[production] |