|
2018-05-28
ยง
|
| 20:14 |
<twentyafterfour> |
Test failures on https://gerrit.wikimedia.org/r/#/c/435825/ are preventing deployment of the fix for a critical deployment blocker (see T195514) 1.32.0-wmf.5 still blocked refs T191051 |
[production] |
| 20:10 |
<twentyafterfour> |
train still held up by test failures: https://gerrit.wikimedia.org/r/#/c/435825/ |
[production] |
| 20:02 |
<elukey> |
restart kafka on kafka1003 as attempt to solve the under-replicated partitions warning |
[production] |
| 19:22 |
<twentyafterfour@tin> |
Synchronized php-1.32.0-wmf.5/extensions/CentralNotice/: sync wmf.5 CentralNotice for AndyRussG (duration: 01m 25s) |
[production] |
| 19:12 |
<elukey> |
roll restart of kafka-mirror maker (main eqiad -> jumbo) on kafka-jumbo* for zookeeper conf updates |
[production] |
| 19:07 |
<twentyafterfour> |
attempting to get the wmf.5 train back on track. Deploying a fix for T195514 (https://gerrit.wikimedia.org/r/c/435292/) to unblock T191051 |
[production] |
| 18:16 |
<elukey> |
restart kafka mirror maker on kafka1012->14 - failed after the last round of kafka restarts |
[production] |
| 17:26 |
<elukey> |
roll restart of kafka on kafka-jumbo* to pick up the new zookeeper settings |
[production] |
| 17:20 |
<gehel@tin> |
Finished deploy [wdqs/wdqs@0e40344]: WDQS updater and GUI (duration: 08m 59s) |
[production] |
| 17:19 |
<elukey> |
restart kafka on kafka1012->23 to pick up the new zookeeper settings |
[production] |
| 17:11 |
<gehel@tin> |
Started deploy [wdqs/wdqs@0e40344]: WDQS updater and GUI |
[production] |
| 17:08 |
<moritzm> |
upgrading mwdebug servers in codfw to hhvm-wikidiff 1.7.0 (HHVM bytecode cache needs to be pruned during rollout) |
[production] |
| 16:48 |
<moritzm> |
upgrading codfw video scalers to hhvm-wikidiff 1.7.0 |
[production] |
| 16:31 |
<elukey> |
roll restart kafka on kafka100[1-3] to pick up new zookeeper settings |
[production] |
| 16:21 |
<elukey> |
zookeeper cluster restart completed (main-eqiad / conf1*) |
[production] |
| 16:18 |
<elukey> |
stop and mask zookeeper on conf1002 |
[production] |
| 16:16 |
<elukey> |
restart prometheus-burrow-exporter on kafkamon* |
[production] |
| 16:12 |
<moritzm> |
upgrading job runners in codfw to hhvm-wikidiff 1.7.0 (HHVM bytecode cache needs to be pruned during rollout) |
[production] |
| 16:02 |
<marostegui> |
Stop MySQL on db2095 for testing - T190704 |
[production] |
| 15:59 |
<elukey> |
swap zookeeper from conf1002 to conf1005 |
[production] |
| 15:56 |
<marostegui> |
Reboot db1124 and db1125 for more testing - T190704 |
[production] |
| 15:49 |
<_joe_> |
uploading cergen 0.2.3 |
[production] |
| 14:44 |
<Amir1> |
EU SWAT is done |
[production] |
| 14:39 |
<ladsgroup@tin> |
Synchronized php-1.32.0-wmf.4/extensions/ArticlePlaceholder: Add config variable to disable SearchHookHandler (T195753) (duration: 01m 21s) |
[production] |
| 14:34 |
<ladsgroup@tin> |
Synchronized wmf-config/InitialiseSettings.php: Disable search integration with Article Placeholder temporarily (T195753) (duration: 01m 20s) |
[production] |
| 14:29 |
<ladsgroup@tin> |
Synchronized php-1.32.0-wmf.5/extensions/ArticlePlaceholder: Add config variable to disable SearchHookHandler (T195753) (duration: 01m 18s) |
[production] |
| 14:23 |
<ladsgroup@tin> |
Synchronized wmf-config/Wikibase-production.php: Disable Special:ItemDisambiguation in Wikidata (T195756) (duration: 01m 20s) |
[production] |
| 14:21 |
<ema> |
cp1045,cp2001,cp3007,cp5001: reboot with intel-microcode T127825 |
[production] |
| 14:00 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:434359|Enable "File mover" flag on zh.wikipedia (T195247)]] (duration: 01m 19s) |
[production] |
| 13:52 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:434508|New protection level on the Hungarian Wikipedia - trusted (T194568)]] (duration: 01m 20s) |
[production] |
| 13:40 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:435628|Revert "Revert "Enable $wgUseRCPatrol on azwiki"" (T194389)]] (duration: 01m 20s) |
[production] |
| 13:34 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:430124|Use uploaded HD logos for yiwikisource (T193562)]] (duration: 01m 19s) |
[production] |
| 13:28 |
<zfilipin@tin> |
Synchronized static/images/project-logos/: SWAT: [[gerrit:430123|Upload new logos for yiwikisource (T193562)]] (duration: 01m 19s) |
[production] |
| 13:21 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:435106|Enable template editor group on newiki (T195557)]] (duration: 01m 21s) |
[production] |
| 13:13 |
<volans> |
restarted pdfrender on scb1002 |
[production] |
| 13:03 |
<moritzm> |
upgrading app servers in codfw to hhvm-wikidiff 1.7.0 (HHVM bytecode cache needs to be pruned during rollout) |
[production] |
| 12:09 |
<arturo> |
T194665 aborrero@install1002:~$ sudo -i reprepro --noskipold -C 'thirdparty/mono-project-stretch' update stretch-wikimedia |
[production] |
| 12:08 |
<arturo> |
T194665 aborrero@install1002:~$ sudo -i reprepro --noskipold -C 'thirdparty/mono-project-jessie' update jessie-wikimedia |
[production] |
| 12:05 |
<arturo> |
aborrero@install1002:~$ sudo -i reprepro --noskipold -C 'thirdparty/mono-project-trusty' update trusty-wikimedia |
[production] |
| 11:31 |
<ema> |
cp1008: reboot with intel-microcode T127825 |
[production] |
| 11:27 |
<moritzm> |
upgrading API servers in codfw to hhvm-wikidiff 1.7.0 (HHVM bytecode cache needs to be pruned during rollout) |
[production] |
| 11:10 |
<marostegui> |
Stop MySQL on db2092 to copy its content to db2094 - T190704 |
[production] |
| 10:36 |
<moritzm> |
upgrading mw1266-mw1275 to hhvm-wikidiff 1.7.0 (HHVM bytecode cache needs to be pruned during rollout) |
[production] |
| 10:14 |
<moritzm> |
upgrading eqiad video scalers to hhvm-wikidiff 1.7.0 |
[production] |
| 10:09 |
<jdrewniak@tin> |
Synchronized portals: Wikimedia Portals Update: [[gerrit:435759|Bumping portals to master (T128546)]] (duration: 01m 20s) |
[production] |
| 10:08 |
<jdrewniak@tin> |
Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:435759|Bumping portals to master (T128546)]] (duration: 01m 21s) |
[production] |
| 09:07 |
<moritzm> |
upgrading mw1299-mw1306 to hhvm-wikidiff 1.7.0 (HHVM bytecode cache needs to be pruned during rollout) |
[production] |
| 09:04 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1083 after alter table (duration: 01m 20s) |
[production] |
| 09:04 |
<marostegui> |
Deploy schema change on s1 primary master (db1052) - T191519 |
[production] |
| 09:00 |
<marostegui> |
Deploy schema change on db1052 (s1 primary master) - T190148 |
[production] |