|
2019-07-02
§
|
| 07:58 |
<moritzm> |
draining restbase2020 for eventual reboot for MDS security updates / OpenJDK security update |
[production] |
| 07:55 |
<ema> |
depool cp2018 and reimage as upload_ats T226637 |
[production] |
| 07:48 |
<moritzm> |
draining restbase2019 for eventual reboot for MDS security updates / OpenJDK security update |
[production] |
| 07:48 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 07:48 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 07:47 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 07:47 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 07:27 |
<jeh> |
clearing out leaked instances that are active and accessible |
[admin-monitoring] |
| 07:20 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Fully repool db1092 (duration: 00m 49s) |
[production] |
| 07:05 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: More traffic to db1092 (duration: 00m 49s) |
[production] |
| 05:48 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Slowly repool db1092 into API (duration: 00m 49s) |
[production] |
| 05:37 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Slowly repool db1092 (duration: 00m 48s) |
[production] |
| 05:23 |
<marostegui> |
Upgrade MySQL and kernel on db1092 |
[production] |
| 05:23 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1092 (duration: 00m 54s) |
[production] |
| 01:40 |
<milimetric> |
deployed refinery, restarted refinery-mediawiki-sqoop-private |
[analytics] |
| 01:39 |
<milimetric@deploy1001> |
Finished deploy [analytics/refinery@b8a496b]: fix private sqoop (duration: 17m 36s) |
[production] |
| 01:21 |
<milimetric@deploy1001> |
Started deploy [analytics/refinery@b8a496b]: fix private sqoop |
[production] |
|
2019-07-01
§
|
| 23:24 |
<James_F> |
Pushing quibble-coverage:0.0.32-1 image |
[releng] |
| 22:30 |
<thcipriani> |
cleaned contint1001 docker images using https://phabricator.wikimedia.org/P8604 |
[releng] |
| 20:40 |
<James_F> |
Rebuilt codehealth jjbs using node10-test with https://gerrit.wikimedia.org/r/519652 |
[releng] |
| 20:33 |
<milimetric@deploy1001> |
Finished deploy [analytics/refinery@4e9894c]: minor, just removing hiwikisource from sqoop list (duration: 01m 33s) |
[production] |
| 20:32 |
<milimetric@deploy1001> |
Started deploy [analytics/refinery@4e9894c]: minor, just removing hiwikisource from sqoop list |
[production] |
| 20:32 |
<milimetric@deploy1001> |
Finished deploy [analytics/refinery@4e9894c]: minor, just removing hiwikisource from sqoop list (duration: 16m 59s) |
[production] |
| 20:29 |
<ottomata> |
removed old refinery deploy caches from an-coord1001 to free up disk space |
[analytics] |
| 20:19 |
<milimetric> |
syncing to hdfs on minor refinery deploy to remove hiwikisource from sqoop lists |
[analytics] |
| 20:15 |
<milimetric@deploy1001> |
Started deploy [analytics/refinery@4e9894c]: minor, just removing hiwikisource from sqoop list |
[production] |
| 19:31 |
<tzatziki> |
removing nine files for legal compliance |
[production] |
| 19:04 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable Special:Homepage for 50% of new users on viwiki (duration: 00m 49s) |
[production] |
| 18:53 |
<urbanecm@deploy1001> |
Synchronized php-1.34.0-wmf.11/extensions/ContentTranslation: SWAT: [[:gerrit:519780|Require only one user group to allow publishing to main namespace]] (T225398) (duration: 00m 49s) |
[production] |
| 18:49 |
<James_F> |
[SecurityCheckPlugin] Revert blocking jobs to php70; move php71+ to experimental |
[releng] |
| 18:49 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SSWAT: [[:gerrit:518260|Dont show cannot publish error to sysop users]] (T225398) (duration: 00m 49s) |
[production] |
| 18:46 |
<urbanecm@deploy1001> |
Synchronized php-1.34.0-wmf.11/extensions/CentralAuth: SWAT: [[:gerrit:519780|Require only one user group to allow publishing to main namespace]] (T225398) (duration: 00m 51s) |
[production] |
| 18:43 |
<James_F> |
Reloading Zuul for mediawiki/tools/phan/SecurityCheckPlugin php72 tests |
[releng] |
| 18:35 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) |
[production] |
| 18:29 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable Special:Homepage on viwiki (T218237) (duration: 00m 49s) |
[production] |
| 18:23 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable EditorJourney on arwiki (T225737) (duration: 00m 49s) |
[production] |
| 17:27 |
<elukey@cumin1001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 17:13 |
<arturo> |
re-creating instance `toolsbeta-test-k8s-master-1` with more CPU for T215531 |
[toolsbeta] |
| 17:03 |
<arturo> |
updated FQDN `toolsbeta-k8s-master.toolsbeta.wmflabs.org` with 172.16.6.9 (the new LB VM) for T215531 |
[toolsbeta] |
| 17:02 |
<arturo> |
re-creating instance `toolsbeta-test-k8s-lb-01` with more CPU for T215531 |
[toolsbeta] |
| 16:58 |
<arturo> |
add puppet prefix `toolsbeta-test-k8s-lb` for T215531 |
[toolsbeta] |
| 16:01 |
<ema> |
pool cp2017 w/ ATS backend T226637 |
[production] |
| 15:51 |
<wm-bot> |
<migr> added hack to update storybook about every hour by running `webservice --backend kubernetes node10 start` and executing the update in the package.json and then sleep for the rest of the hour |
[tools.wikibase-databridge-storybook] |
| 15:42 |
<moritzm> |
draining restbase2018 for eventual reboot for MDS kernel updates |
[production] |
| 15:37 |
<elukey@cumin1001> |
END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) |
[production] |
| 15:36 |
<elukey@cumin1001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 15:31 |
<moritzm> |
draining restbase2017 for eventual reboot for MDS kernel updates |
[production] |
| 15:27 |
<elukey@cumin1001> |
END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) |
[production] |
| 15:27 |
<elukey@cumin1001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 15:19 |
<moritzm> |
draining restbase2016 for eventual reboot for MDS kernel updates |
[production] |