|
2020-05-07
ยง
|
| 21:51 |
<bstorm_> |
rebuilding the docker images for Toolforge k8s |
[tools] |
| 20:40 |
<ryankemper@cumin2001> |
END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) |
[production] |
| 20:37 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 20:10 |
<otto@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: wgEventLoggingStreamNames: set initial stream names, as yet unused - T238230 (duration: 01m 07s) |
[production] |
| 19:52 |
<Krinkle> |
Reloading Zuul to deploy https://gerrit.wikimedia.org/r/595026 |
[releng] |
| 19:12 |
<brennen@deploy1001> |
rebuilt and synchronized wikiversions files: Revert group2 wikis to 1.35.0-wmf.30 |
[production] |
| 19:09 |
<brennen> |
rolling 1.35.0-wmf.31 back to group1 |
[production] |
| 19:09 |
<XioNoX> |
Upgrade Routinator 3000 to 0.7.0 on rpki1001 - T252010 |
[production] |
| 19:05 |
<brennen@deploy1001> |
rebuilt and synchronized wikiversions files: all wikis to 1.35.0-wmf.31 |
[production] |
| 19:03 |
<bstorm_> |
toollabs-webservice 0.69 now pushed to the Toolforge bastions |
[tools] |
| 18:57 |
<bstorm_> |
pushing new toollabs-webservice package v0.69 to the tools repos |
[tools] |
| 18:25 |
<ppchelko@deploy1001> |
Finished deploy [changeprop/deploy@383fba5]: Enable both purging types T252142 (duration: 01m 17s) |
[production] |
| 18:23 |
<ppchelko@deploy1001> |
Started deploy [changeprop/deploy@383fba5]: Enable both purging types T252142 |
[production] |
| 18:15 |
<Urbanecm> |
Morning SWAT done |
[production] |
| 18:14 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 899c175: Update project icons to refreshed SVGs (T249047; part 2/2) (duration: 01m 06s) |
[production] |
| 18:13 |
<urbanecm@deploy1001> |
Synchronized static/images/mobile/copyright/: SWAT: 899c175: Update project icons to refreshed SVGs (T249047; part 1/2) (duration: 01m 08s) |
[production] |
| 18:08 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 54bd2f1: Add the investigate right to the checkuser group on testwiki (T251932) (duration: 01m 08s) |
[production] |
| 17:50 |
<bsitzmann@deploy1001> |
helmfile [CODFW] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . |
[production] |
| 17:46 |
<bsitzmann@deploy1001> |
helmfile [EQIAD] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . |
[production] |
| 17:44 |
<bsitzmann@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'wikifeeds' for release 'staging' . |
[production] |
| 17:44 |
<otto@deploy1001> |
Finished deploy [analytics/refinery@4a2c530]: (no justification provided) (duration: 05m 31s) |
[production] |
| 17:39 |
<ottomata> |
deploying fix to refinery bin/camus CamusPartitionChecker when using dynamic stream configs |
[analytics] |
| 17:38 |
<otto@deploy1001> |
Started deploy [analytics/refinery@4a2c530]: (no justification provided) |
[production] |
| 17:18 |
<ejegg> |
updated payments-wiki from afb84cc391 to dabba1804c |
[production] |
| 16:49 |
<joal> |
Restart and babysit mediawiki-history-denormalize-wf-2020-04 |
[analytics] |
| 16:46 |
<hnowlan@deploy1001> |
Finished deploy [changeprop/deploy@cd1386e]: Rollback varnish consumption (duration: 01m 05s) |
[production] |
| 16:45 |
<hnowlan@deploy1001> |
Started deploy [changeprop/deploy@cd1386e]: Rollback varnish consumption |
[production] |
| 16:42 |
<mvolz@deploy1001> |
helmfile [EQIAD] Ran 'sync' command on namespace 'citoid' for release 'production' . |
[production] |
| 16:37 |
<elukey> |
roll restart of all the nodemanagers on the hadoop cluster to pick up new jvm settings |
[analytics] |
| 16:36 |
<mvolz@deploy1001> |
helmfile [CODFW] Ran 'sync' command on namespace 'citoid' for release 'production' . |
[production] |
| 16:33 |
<arturo> |
livehack toolsbeta-puppetmaster-03 with https://gerrit.wikimedia.org/r/c/operations/puppet/+/594945 (T251297 and T250866) |
[toolsbeta] |
| 16:32 |
<cmjohnson@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 16:30 |
<cmjohnson@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 16:29 |
<mvolz@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'citoid' for release 'staging' . |
[production] |
| 16:27 |
<cmjohnson@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 16:27 |
<cmjohnson@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 16:26 |
<cmjohnson@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 16:26 |
<hnowlan@deploy1001> |
Finished deploy [changeprop/deploy@cd1386e]: Enabling consumption of purges topic (duration: 01m 45s) |
[production] |
| 16:25 |
<cmjohnson@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 16:24 |
<hnowlan@deploy1001> |
Started deploy [changeprop/deploy@cd1386e]: Enabling consumption of purges topic |
[production] |
| 16:23 |
<hnowlan@deploy1001> |
Finished deploy [changeprop/deploy@6c65779]: Enabling consumption of purges topic (duration: 00m 24s) |
[production] |
| 16:23 |
<hnowlan@deploy1001> |
Started deploy [changeprop/deploy@6c65779]: Enabling consumption of purges topic |
[production] |
| 16:20 |
<James_F> |
Deleted integration-cumin.integration.eqiad.wmflabs for last part of T236576 |
[releng] |
| 16:17 |
<James_F> |
Shut down integration-cumin-02, non-functional buster cumin host. |
[releng] |
| 16:15 |
<James_F> |
Shutting off integration-cumin for T236576 |
[releng] |
| 16:14 |
<James_F> |
Dropped integration-cumin (172.16.1.103) from WMCS integration cumin master list |
[releng] |
| 16:11 |
<James_F> |
Shutting down integration-agent-stretch-1001, unused. |
[releng] |
| 16:06 |
<hashar> |
integration-agent-stretch-1001 : clearing out /var/cache/pbuilder/build |
[releng] |
| 15:59 |
<mvolz@deploy1001> |
helmfile [EQIAD] Ran 'sync' command on namespace 'citoid' for release 'production' . |
[production] |
| 15:51 |
<mvolz@deploy1001> |
helmfile [CODFW] Ran 'sync' command on namespace 'citoid' for release 'production' . |
[production] |