|
2021-12-01
§
|
| 09:16 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on kubetcd2005.codfw.wmnet with reason: Switch to DRBD for migration |
[production] |
| 09:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'After maintenance db1096:3316 (T277354)', diff saved to https://phabricator.wikimedia.org/P17918 and previous config saved to /var/cache/conftool/dbconfig/20211201-091110-marostegui.json |
[production] |
| 09:09 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1096:3316 (T277354)', diff saved to https://phabricator.wikimedia.org/P17917 and previous config saved to /var/cache/conftool/dbconfig/20211201-090948-marostegui.json |
[production] |
| 09:09 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1096.eqiad.wmnet with reason: Maintenance T277354 |
[production] |
| 09:09 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 4:00:00 on db1096.eqiad.wmnet with reason: Maintenance T277354 |
[production] |
| 09:03 |
<vgutierrez> |
rolling restart of haproxy and varnish on O:cache::text_haproxy and O:cache::upload_haproxy - T290005 |
[production] |
| 08:56 |
<moritzm> |
draining primary/secondary instance off ganeti2010 T296622 |
[production] |
| 08:51 |
<oblivian@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 08:41 |
<oblivian@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 08:32 |
<oblivian@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 06:20 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2141.codfw.wmnet with reason: Maintenance T277354 |
[production] |
| 06:20 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 4:00:00 on db2141.codfw.wmnet with reason: Maintenance T277354 |
[production] |
| 06:20 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2124.codfw.wmnet with reason: Maintenance T277354 |
[production] |
| 06:20 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 4:00:00 on db2124.codfw.wmnet with reason: Maintenance T277354 |
[production] |
| 06:19 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2117.codfw.wmnet with reason: Maintenance T277354 |
[production] |
| 06:19 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 4:00:00 on db2117.codfw.wmnet with reason: Maintenance T277354 |
[production] |
| 00:35 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 00:35 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 00:32 |
<catrope@deploy1002> |
Synchronized php-1.38.0-wmf.9/extensions/GrowthExperiments/includes/NewcomerTasks/NewcomerTasksUserOptionsLookup.php: Backport: [[gerrit:742548|Newcomer tasks: Fix filtering of non-existent task types (T296366)]] (duration: 00m 56s) |
[production] |
| 00:10 |
<catrope@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:742817|Enable A/B test enrollment instrumentation. (T292587)]] (duration: 00m 56s) |
[production] |
| 00:08 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 00:07 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
|
2021-11-30
§
|
| 23:59 |
<dzahn@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'miscweb' for release 'main' . |
[production] |
| 23:57 |
<mutante> |
deploy1002 - kube_env miscweb staging ; helmfile -e staging destroy |
[production] |
| 23:56 |
<dzahn@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'miscweb' for release 'main' . |
[production] |
| 23:46 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 23:45 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 23:09 |
<mutante> |
gerrit - added Majavah to wmf-deployment group for T296777 |
[production] |
| 22:30 |
<krinkle@deploy1002> |
Finished deploy [integration/docroot@2af7007]: Ia89b6591639e5 (duration: 00m 09s) |
[production] |
| 22:30 |
<krinkle@deploy1002> |
Started deploy [integration/docroot@2af7007]: Ia89b6591639e5 |
[production] |
| 22:21 |
<mutante> |
welcome Majavah to MediaWiki deployers (T296777) |
[production] |
| 20:09 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 5443b78f197b782238632966891d721859733a74: uzwiki: Deploy Growth features to newcomers (T294245) (duration: 00m 57s) |
[production] |
| 18:09 |
<legoktm> |
uploaded php-yaml for component/php72 (T296331) |
[production] |
| 18:08 |
<vgutierrez> |
restart haproxy on cp3064 - T290005 |
[production] |
| 17:44 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'Repool db1163 fully', diff saved to https://phabricator.wikimedia.org/P17912 and previous config saved to /var/cache/conftool/dbconfig/20211130-174434-jynus.json |
[production] |
| 17:39 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'Repool db1163 at 50%', diff saved to https://phabricator.wikimedia.org/P17911 and previous config saved to /var/cache/conftool/dbconfig/20211130-173935-jynus.json |
[production] |
| 17:35 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'Repool db1163 at 25%', diff saved to https://phabricator.wikimedia.org/P17910 and previous config saved to /var/cache/conftool/dbconfig/20211130-173517-jynus.json |
[production] |
| 17:34 |
<moritzm> |
installing libvorbis security updates |
[production] |
| 17:15 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'Repool db1163 at 5%', diff saved to https://phabricator.wikimedia.org/P17908 and previous config saved to /var/cache/conftool/dbconfig/20211130-171550-jynus.json |
[production] |
| 17:00 |
<jynus> |
move db1139:s1 under db1118 |
[production] |
| 16:57 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'Depool db1163 fully', diff saved to https://phabricator.wikimedia.org/P17907 and previous config saved to /var/cache/conftool/dbconfig/20211130-165718-jynus.json |
[production] |
| 16:29 |
<XioNoX> |
Move cr2-codfw lumen transit link to BO cable - T289241 |
[production] |
| 16:26 |
<XioNoX> |
Move cr2-codfw eqord link to BO cable - T289241 |
[production] |
| 16:23 |
<XioNoX> |
Move cr2-codfw pfw3 link to BO cable - T289241 |
[production] |
| 16:20 |
<Emperor> |
reboot ms-be2059 to fix device enumeration order re T295563 |
[production] |
| 16:14 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'Depool db1163 at 25%', diff saved to https://phabricator.wikimedia.org/P17906 and previous config saved to /var/cache/conftool/dbconfig/20211130-161457-jynus.json |
[production] |
| 16:13 |
<XioNoX> |
cr2-codfw bounce fpc 1 pic 0 (vrrp backup) - T289241 |
[production] |
| 16:07 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'Depool db1163 at 50%', diff saved to https://phabricator.wikimedia.org/P17905 and previous config saved to /var/cache/conftool/dbconfig/20211130-160748-jynus.json |
[production] |
| 16:06 |
<bblack> |
lvs2007 - repooling into service |
[production] |
| 16:01 |
<bblack> |
lvs2007 - depooling for network maint - do not push LVS config changes please! |
[production] |