|
2018-01-17
§
|
| 07:48 |
<elukey> |
restart varnish backend on cp4024 (ton of 503s, icinga alerting for mailbox lag) |
[production] |
| 07:46 |
<oblivian@neodymium> |
conftool action : set/pooled=inactive; selector: cluster=appserver,name=mw12([0-1][0-9]|20)\.eqiad\.wmnet |
[production] |
| 07:45 |
<_joe_> |
depooling mw1209-1220 from the appserver cluster for decommissioning, T185004 |
[production] |
| 06:47 |
<marostegui> |
Remove labsdb1001 and labsdb1003 from tendril - T184832 |
[production] |
| 06:40 |
<marostegui> |
Stop MySQL on labsdb1001 (already dead) and labsdb1003 - T184832 |
[production] |
| 06:29 |
<marostegui> |
Stop replication in sync on db1089 and s1 codfw master (db2048) - T162807 |
[production] |
| 06:28 |
<marostegui> |
Deploy schema change on db1104 - T174569 |
[production] |
| 06:21 |
<marostegui> |
Upgrade mariadb and kernel on db1104 |
[production] |
| 06:20 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1104 - T174569 (duration: 01m 14s) |
[production] |
| 02:31 |
<l10nupdate@tin> |
scap sync-l10n completed (1.31.0-wmf.16) (duration: 07m 11s) |
[production] |
| 00:28 |
<ebernhardson@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: T182616 Remove cirrus AB test config for hewiki (duration: 01m 09s) |
[production] |
| 00:26 |
<ebernhardson@tin> |
Synchronized fc-list: SWAT: T184664 Updating fonts list and sorting it (duration: 01m 12s) |
[production] |
| 00:21 |
<ebernhardson@tin> |
Synchronized fc-list: SWAT: T184664 Updating fonts list and sorting it (duration: 01m 12s) |
[production] |
| 00:10 |
<ebernhardson@tin> |
Synchronized php-1.31.0-wmf.16/extensions/WikimediaEvents/modules/all/ext.wikimediaEvents.searchSatisfaction.js: SWAT: T182616 Turn off cirrus AB test on hewiki (duration: 01m 12s) |
[production] |
| 00:08 |
<ebernhardson@tin> |
Synchronized php-1.31.0-wmf.17/extensions/WikimediaEvents/modules/all/ext.wikimediaEvents.searchSatisfaction.js: SWAT: T182616 Turn off cirrus AB test on hewiki (duration: 01m 14s) |
[production] |
|
2018-01-16
§
|
| 22:57 |
<niharika29@tin> |
Finished deploy [scholarships/scholarships@728d203]: Update privacy statement and delete invalidated translation files. T184659 (duration: 00m 02s) |
[production] |
| 22:57 |
<niharika29@tin> |
Started deploy [scholarships/scholarships@728d203]: Update privacy statement and delete invalidated translation files. T184659 |
[production] |
| 22:53 |
<thcipriani@tin> |
rebuilt and synchronized wikiversions files: group0 to 1.31.0-wmf.17 |
[production] |
| 22:40 |
<mobrovac@tin> |
Synchronized wmf-config/InitialiseSettings.php: Use EventBus for htmlCacheUpdate jobs for all wikis but en, commons and wikidata - T182023 (duration: 01m 12s) |
[production] |
| 22:39 |
<ppchelko@tin> |
Finished deploy [cpjobqueue/deploy@19b9bdd]: Switch htmlCacheUpdates for all but en, commons, wikidata T182023 (duration: 00m 35s) |
[production] |
| 22:39 |
<ppchelko@tin> |
Started deploy [cpjobqueue/deploy@19b9bdd]: Switch htmlCacheUpdates for all but en, commons, wikidata T182023 |
[production] |
| 22:19 |
<thcipriani@tin> |
Synchronized php-1.31.0-wmf.17/extensions/WikimediaMessages/WikimediaMessages.hooks.php: [[gerrit:404583|Update access to ORES isModelEnabled()]] (duration: 01m 13s) |
[production] |
| 22:19 |
<ottomata> |
apt-get install librdkafka1=0.9.4-1~jessie1 librdkafka++1=0.9.4-1~jessie1 on scb* to put librdkafka back at node-rdkafka compat version (somehow this was upgraded yesterday...very dangerous!!) |
[production] |
| 22:16 |
<thcipriani@tin> |
Finished scap: testwiki to php-1.31.0-wmf.17 and rebuild l10n cache (duration: 25m 45s) |
[production] |
| 21:50 |
<thcipriani@tin> |
Started scap: testwiki to php-1.31.0-wmf.17 and rebuild l10n cache |
[production] |
| 21:15 |
<andrewbogott> |
rebooting labvirt1014 and 1015 |
[production] |
| 21:04 |
<thcipriani@tin> |
rebuilt and synchronized wikiversions files: all wikis to 1.31.0-wmf.16 |
[production] |
| 20:59 |
<andrewbogott> |
rebooting labvirt1013 |
[production] |
| 20:42 |
<demon@tin> |
Finished scap: wmf.17 files, no bootstrap of i18n tho (x2) (duration: 06m 33s) |
[production] |
| 20:35 |
<demon@tin> |
Started scap: wmf.17 files, no bootstrap of i18n tho (x2) |
[production] |
| 20:34 |
<demon@tin> |
scap aborted: wmf.17 files, no bootstrap of i18n tho (duration: 08m 59s) |
[production] |
| 20:34 |
<andrewbogott> |
rebooting labvirt1011 |
[production] |
| 20:32 |
<herron> |
re-enabling puppet agents |
[production] |
| 20:25 |
<demon@tin> |
Started scap: wmf.17 files, no bootstrap of i18n tho |
[production] |
| 20:24 |
<herron> |
temporarily disabling puppet agents while troubleshooting puppet crl |
[production] |
| 20:07 |
<thcipriani@tin> |
rebuilt and synchronized wikiversions files: group1 to 1.31.0-wmf.16 |
[production] |
| 20:03 |
<andrewbogott> |
rebooting labvirt1009 |
[production] |
| 19:46 |
<andrewbogott> |
rebooting labvirt1008 |
[production] |
| 19:46 |
<thcipriani@tin> |
Synchronized php-1.31.0-wmf.16/includes/Storage/RevisionStore.php: [[gerrit:403930|RevisionStore, fix loadSlotContent with no $blobFlags]] T184749 (duration: 01m 13s) |
[production] |
| 19:30 |
<twentyafterfour> |
restarted wikibugs (several attempts, eventually it worked) |
[production] |
| 18:50 |
<chasemp> |
reboot labvirt1020 |
[production] |
| 18:44 |
<chasemp> |
reboot labvirt1019 |
[production] |
| 18:35 |
<andrewbogott> |
rebooting labvirt1007 |
[production] |
| 18:30 |
<arlolra@tin> |
Finished deploy [parsoid/deploy@1026fd2]: Updating Parsoid to 231bfff (duration: 13m 13s) |
[production] |
| 18:25 |
<herron> |
removing ganeti VM puppetcompiler1001 |
[production] |
| 18:19 |
<moritzm> |
rebooting labmon1002 for kernel security update |
[production] |
| 18:17 |
<arlolra@tin> |
Started deploy [parsoid/deploy@1026fd2]: Updating Parsoid to 231bfff |
[production] |
| 17:59 |
<moritzm> |
rebooting labnet100[34] and labcontrol100[34] for kernel security update |
[production] |
| 17:52 |
<herron> |
re-enabled puppet agents |
[production] |
| 17:50 |
<andrewbogott> |
rebooting labvirt1005 |
[production] |