|
2020-05-08
§
|
| 07:07 |
<mutante> |
phabricator rmdir /var/run/phd/pid - empty and now unused |
[production] |
| 07:01 |
<mutante> |
phabricator-prod-1001 - removing cron for public task dump (though puppet should have removed it) |
[devtools] |
| 07:01 |
<moritzm> |
installing php5 security updates |
[production] |
| 05:27 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 05:24 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 05:10 |
<marostegui> |
Upgrade pc1010 |
[production] |
| 02:07 |
<Krinkle> |
cs access #cvn-wp-en add Examknow voiced |
[cvn] |
| 01:57 |
<wm-bot> |
<samwilson> Switched to canonical domain name. |
[tools.svgtranslate-test] |
| 00:30 |
<brennen@deploy1001> |
rebuilt and synchronized wikiversions files: Revert all wikis except test to 1.35.0-wmf.30 for T252179 |
[production] |
| 00:19 |
<brennen> |
rolling 1.35.0-wmf.31 train back to group0 for T252179 |
[production] |
|
2020-05-07
§
|
| 22:36 |
<brennen@deploy1001> |
rebuilt and synchronized wikiversions files: all wikis to 1.35.0-wmf.31 |
[production] |
| 22:31 |
<brennen@deploy1001> |
Synchronized php-1.35.0-wmf.31/extensions/Scribunto/includes/engines/LuaCommon/TitleLibrary.php: [[gerrit:595054|Handle RevisionAccessException with try-catch (T252156)]] (duration: 01m 08s) |
[production] |
| 21:51 |
<bstorm_> |
rebuilding the docker images for Toolforge k8s |
[tools] |
| 20:40 |
<ryankemper@cumin2001> |
END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) |
[production] |
| 20:37 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 20:10 |
<otto@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: wgEventLoggingStreamNames: set initial stream names, as yet unused - T238230 (duration: 01m 07s) |
[production] |
| 19:52 |
<Krinkle> |
Reloading Zuul to deploy https://gerrit.wikimedia.org/r/595026 |
[releng] |
| 19:12 |
<brennen@deploy1001> |
rebuilt and synchronized wikiversions files: Revert group2 wikis to 1.35.0-wmf.30 |
[production] |
| 19:09 |
<brennen> |
rolling 1.35.0-wmf.31 back to group1 |
[production] |
| 19:09 |
<XioNoX> |
Upgrade Routinator 3000 to 0.7.0 on rpki1001 - T252010 |
[production] |
| 19:05 |
<brennen@deploy1001> |
rebuilt and synchronized wikiversions files: all wikis to 1.35.0-wmf.31 |
[production] |
| 19:03 |
<bstorm_> |
toollabs-webservice 0.69 now pushed to the Toolforge bastions |
[tools] |
| 18:57 |
<bstorm_> |
pushing new toollabs-webservice package v0.69 to the tools repos |
[tools] |
| 18:25 |
<ppchelko@deploy1001> |
Finished deploy [changeprop/deploy@383fba5]: Enable both purging types T252142 (duration: 01m 17s) |
[production] |
| 18:23 |
<ppchelko@deploy1001> |
Started deploy [changeprop/deploy@383fba5]: Enable both purging types T252142 |
[production] |
| 18:15 |
<Urbanecm> |
Morning SWAT done |
[production] |
| 18:14 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 899c175: Update project icons to refreshed SVGs (T249047; part 2/2) (duration: 01m 06s) |
[production] |
| 18:13 |
<urbanecm@deploy1001> |
Synchronized static/images/mobile/copyright/: SWAT: 899c175: Update project icons to refreshed SVGs (T249047; part 1/2) (duration: 01m 08s) |
[production] |
| 18:08 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 54bd2f1: Add the investigate right to the checkuser group on testwiki (T251932) (duration: 01m 08s) |
[production] |
| 17:50 |
<bsitzmann@deploy1001> |
helmfile [CODFW] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . |
[production] |
| 17:46 |
<bsitzmann@deploy1001> |
helmfile [EQIAD] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . |
[production] |
| 17:44 |
<bsitzmann@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'wikifeeds' for release 'staging' . |
[production] |
| 17:44 |
<otto@deploy1001> |
Finished deploy [analytics/refinery@4a2c530]: (no justification provided) (duration: 05m 31s) |
[production] |
| 17:39 |
<ottomata> |
deploying fix to refinery bin/camus CamusPartitionChecker when using dynamic stream configs |
[analytics] |
| 17:38 |
<otto@deploy1001> |
Started deploy [analytics/refinery@4a2c530]: (no justification provided) |
[production] |
| 17:18 |
<ejegg> |
updated payments-wiki from afb84cc391 to dabba1804c |
[production] |
| 16:49 |
<joal> |
Restart and babysit mediawiki-history-denormalize-wf-2020-04 |
[analytics] |
| 16:46 |
<hnowlan@deploy1001> |
Finished deploy [changeprop/deploy@cd1386e]: Rollback varnish consumption (duration: 01m 05s) |
[production] |
| 16:45 |
<hnowlan@deploy1001> |
Started deploy [changeprop/deploy@cd1386e]: Rollback varnish consumption |
[production] |
| 16:42 |
<mvolz@deploy1001> |
helmfile [EQIAD] Ran 'sync' command on namespace 'citoid' for release 'production' . |
[production] |
| 16:37 |
<elukey> |
roll restart of all the nodemanagers on the hadoop cluster to pick up new jvm settings |
[analytics] |
| 16:36 |
<mvolz@deploy1001> |
helmfile [CODFW] Ran 'sync' command on namespace 'citoid' for release 'production' . |
[production] |
| 16:33 |
<arturo> |
livehack toolsbeta-puppetmaster-03 with https://gerrit.wikimedia.org/r/c/operations/puppet/+/594945 (T251297 and T250866) |
[toolsbeta] |
| 16:32 |
<cmjohnson@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 16:30 |
<cmjohnson@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 16:29 |
<mvolz@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'citoid' for release 'staging' . |
[production] |
| 16:27 |
<cmjohnson@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 16:27 |
<cmjohnson@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 16:26 |
<cmjohnson@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 16:26 |
<hnowlan@deploy1001> |
Finished deploy [changeprop/deploy@cd1386e]: Enabling consumption of purges topic (duration: 01m 45s) |
[production] |