|
2018-07-05
ยง
|
| 21:55 |
<twentyafterfour> |
whitelist office IPs in phabricator throttle |
[production] |
| 21:30 |
<bstorm_> |
disabled unused P440ar RAID controller on labvirt1020 |
[production] |
| 21:15 |
<bstorm_> |
disabled unused RAID controller on labvirt1019 |
[production] |
| 19:30 |
<gehel> |
repool wdqs1003, lag almost completely recovered |
[production] |
| 19:07 |
<ebernhardson> |
T194678 un-pause cirrussearch writes to codfw |
[production] |
| 18:58 |
<gehel> |
depooling wdqs1003 to help it recover update lag |
[production] |
| 18:55 |
<ebernhardson> |
T194678 pause cirrussearch writes to codfw to check how kafka+mirrormaker responds |
[production] |
| 18:48 |
<gehel> |
restarting blazegraph on wdqs1003 (updater lag) |
[production] |
| 18:21 |
<thcipriani@deploy1001> |
Synchronized wmf-config/Wikibase-production.php: SWAT: [[gerrit:443939|Set dispatchLagToMaxLagFactor to 60 for wikidata]] T194950 (duration: 00m 51s) |
[production] |
| 18:14 |
<thcipriani@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:443937|Add "L" as an alias of Lexeme namespace in wikidata]] T195493 (duration: 00m 59s) |
[production] |
| 18:10 |
<gehel@deploy1001> |
Finished deploy [wdqs/wdqs@228f9c5]: new version of wdqs GUI and blazegraph (wdqs1010 only) (duration: 00m 28s) |
[production] |
| 18:09 |
<gehel@deploy1001> |
Started deploy [wdqs/wdqs@228f9c5]: new version of wdqs GUI and blazegraph (wdqs1010 only) |
[production] |
| 16:13 |
<jynus> |
stop and reimage db2045 |
[production] |
| 15:34 |
<mobrovac@deploy1001> |
Finished scap: Set the Accept-Language header explicitly when making requests to the REST API - T198186 (duration: 30m 50s) |
[production] |
| 15:26 |
<elukey> |
upgrade (without jvm restart) prometheus-jmx-exporter on the analytics node listed in debmonitor still not running the last version |
[production] |
| 15:03 |
<mobrovac@deploy1001> |
Started scap: Set the Accept-Language header explicitly when making requests to the REST API - T198186 |
[production] |
| 15:02 |
<jynus> |
stop and reimage db2040 |
[production] |
| 15:00 |
<marostegui> |
Optimize dewiki.logging on db1082 with replication, this will generate lag on s5 on labsdb hosts/script load irssinotifier - T197459 |
[production] |
| 14:54 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1082 for alter table (duration: 00m 50s) |
[production] |
| 14:50 |
<moritzm> |
installing php security updates on netmon1002 |
[production] |
| 14:16 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1097:3315 after alter table (duration: 00m 50s) |
[production] |
| 14:15 |
<moritzm> |
installing dbus updates from stretch point release |
[production] |
| 14:09 |
<zfilipin@deploy1001> |
Synchronized php-1.32.0-wmf.10/extensions/EventBus/: SWAT: [[gerrit:443932| Dont specify the comment if it is an empty string]] (duration: 00m 51s) |
[production] |
| 14:03 |
<jynus> |
stop and reimage db2052 |
[production] |
| 13:55 |
<moritzm> |
installing glibc updates from stretch point release |
[production] |
| 13:55 |
<zfilipin@deploy1001> |
Synchronized static/images/project-logos: SWAT: [[gerrit:443933|Revert "Change logo for eswiki temporarily" (T198846)]] (duration: 00m 50s) |
[production] |
| 13:50 |
<moritzm> |
installing postgresql security updates on maps cluster |
[production] |
| 13:41 |
<aaron@deploy1001> |
Synchronized wmf-config/mc.php: Make test wikis just write to both nutcracker and mcrouter (duration: 00m 50s) |
[production] |
| 13:32 |
<zfilipin@deploy1001> |
Synchronized wmf-config: SWAT: [[gerrit:443686|Enable ORES wp10, draftquality on draft ns (118) for enwiki (T198768)]] (duration: 00m 51s) |
[production] |
| 13:26 |
<zfilipin@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:442142| Replace Tidy with RemexHtml everywhere (T175706)]] (duration: 00m 50s) |
[production] |
| 13:18 |
<marostegui> |
Optimize dewiki.logging on db1113:3315 - T197459 |
[production] |
| 13:17 |
<marostegui> |
Deploy schema change on db1113:3315 T146591 T197891 T196379 |
[production] |
| 13:12 |
<ladsgroup@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:443387|Set $wgChangeTagsSchemaMigrationStage to write both for Wikivoyage, Wikibooks (T194165)]] (duration: 00m 52s) |
[production] |
| 11:30 |
<moritzm> |
installing python-mimeparse updates from jessie point release |
[production] |
| 11:06 |
<moritzm> |
rebooting contint1001 for kernel update/enabling microcode |
[production] |
| 11:05 |
<hashar> |
Stopping Jenkins CI for kernel upgrade on contint1001 |
[production] |
| 10:31 |
<moritzm> |
installing ncurses security updates on jessie |
[production] |
| 09:50 |
<marostegui> |
Optimize dewiki.logging on db1097:3315 - T197459 |
[production] |
| 09:50 |
<marostegui> |
Deploy schema change on db1097:3315 T146591 T197891 T196379 |
[production] |
| 09:50 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1097:3315 for alter table (duration: 00m 50s) |
[production] |
| 09:42 |
<moritzm> |
rebooting ms-be1013/1016/1028/1040 as microcode update canaries |
[production] |
| 09:40 |
<mobrovac@deploy1001> |
Finished deploy [zotero/translators@d6702bc]: String.includes() polyfill for Sveriges radio translator - T187023 (duration: 00m 08s) |
[production] |
| 09:40 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1096:3315 after alter table (duration: 00m 52s) |
[production] |
| 09:40 |
<mobrovac@deploy1001> |
Started deploy [zotero/translators@d6702bc]: String.includes() polyfill for Sveriges radio translator - T187023 |
[production] |
| 09:25 |
<moritzm> |
rebooting ms-fe1005 as microcode update canary |
[production] |
| 09:11 |
<godog> |
drain restbase200[234] and restart cassandra to pick up updated certificates |
[production] |
| 08:57 |
<moritzm> |
draining restbase1007 for reboot to pick up Intel microcode update |
[production] |
| 08:42 |
<moritzm> |
draining restbase2010 for reboot to pick up Intel microcode update |
[production] |
| 08:26 |
<godog> |
add graphite2003 to carbon-c-relay frontend on graphite1001 - T196483 |
[production] |
| 08:12 |
<moritzm> |
draining restbase2001 for reboot to pick up Intel microcode update |
[production] |