3401-3450 of 10000 results (57ms)
2022-05-04 ยง
08:50 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1171.eqiad.wmnet with reason: Maintenance [production]
08:45 <btullis@cumin1001> START - Cookbook sre.hosts.reboot-single for host an-coord1002.eqiad.wmnet [production]
08:43 <taavi@deploy1002> Synchronized php-1.39.0-wmf.9/extensions/TitleBlacklist/extension.json: Backport: [[gerrit:789095|wmf.9 HACK: add forward class alias for TitleBlacklistEntry too (T307513)]] (duration: 00m 49s) [production]
08:43 <taavi@deploy1002> Synchronized php-1.39.0-wmf.9/extensions/TitleBlacklist/includes/TitleBlacklistEntry.php: Backport: [[gerrit:789095|wmf.9 HACK: add forward class alias for TitleBlacklistEntry too (T307513)]] (duration: 00m 50s) [production]
08:41 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1110', diff saved to https://phabricator.wikimedia.org/P27417 and previous config saved to /var/cache/conftool/dbconfig/20220504-084142-ladsgroup.json [production]
08:41 <hashar@deploy1002> deploy-promote aborted: (duration: 00m 11s) [production]
08:39 <taavi@deploy1002> Synchronized php-1.39.0-wmf.10/extensions/TitleBlacklist: Backport: [[gerrit:788855|Add a class_alias for TitleBlacklistEntry too (T307513)]] (duration: 00m 50s) [production]
08:36 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
08:35 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
08:35 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
08:34 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
08:29 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
08:29 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
08:29 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
08:28 <hashar@deploy1002> rebuilt and synchronized wikiversions files: Revert "group0 wikis to 1.39.0-wmf.10 - T305216 T307513 [production]
08:28 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
08:26 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1110', diff saved to https://phabricator.wikimedia.org/P27416 and previous config saved to /var/cache/conftool/dbconfig/20220504-082637-ladsgroup.json [production]
08:23 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
08:22 <hashar@deploy1002> rebuilt and synchronized wikiversions files: group0 wikis to 1.39.0-wmf.10 refs T305216 [production]
08:22 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
08:22 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
08:18 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
08:15 <hashar@deploy1002> Synchronized php-1.39.0-wmf.10/extensions/CentralAuth/includes/Special/SpecialGlobalGroupMembership.php: SpecialGlobalGroupMembership: do not call core hooks - T307518 (duration: 01m 09s) [production]
08:11 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1110 (T307525)', diff saved to https://phabricator.wikimedia.org/P27415 and previous config saved to /var/cache/conftool/dbconfig/20220504-081131-ladsgroup.json [production]
08:10 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on 10 hosts with reason: Maintenance [production]
08:10 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on 10 hosts with reason: Maintenance [production]
08:10 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2121.codfw.wmnet with reason: Maintenance [production]
08:10 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2121.codfw.wmnet with reason: Maintenance [production]
08:10 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1136 (T307525)', diff saved to https://phabricator.wikimedia.org/P27414 and previous config saved to /var/cache/conftool/dbconfig/20220504-081015-ladsgroup.json [production]
08:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1110 (T307525)', diff saved to https://phabricator.wikimedia.org/P27413 and previous config saved to /var/cache/conftool/dbconfig/20220504-080509-ladsgroup.json [production]
08:05 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1110.eqiad.wmnet with reason: Maintenance [production]
08:05 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1110.eqiad.wmnet with reason: Maintenance [production]
08:00 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1150.eqiad.wmnet with reason: Maintenance [production]
08:00 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1150.eqiad.wmnet with reason: Maintenance [production]
07:59 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on 8 hosts with reason: Maintenance [production]
07:59 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on 8 hosts with reason: Maintenance [production]
07:58 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2123.codfw.wmnet with reason: Maintenance [production]
07:58 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2123.codfw.wmnet with reason: Maintenance [production]
07:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1136', diff saved to https://phabricator.wikimedia.org/P27412 and previous config saved to /var/cache/conftool/dbconfig/20220504-075510-ladsgroup.json [production]
07:54 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on dbstore1003.eqiad.wmnet with reason: Maintenance [production]
07:54 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on dbstore1003.eqiad.wmnet with reason: Maintenance [production]
07:40 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1136', diff saved to https://phabricator.wikimedia.org/P27411 and previous config saved to /var/cache/conftool/dbconfig/20220504-074005-ladsgroup.json [production]
07:25 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1136 (T307525)', diff saved to https://phabricator.wikimedia.org/P27410 and previous config saved to /var/cache/conftool/dbconfig/20220504-072500-ladsgroup.json [production]
07:10 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1136 (T307525)', diff saved to https://phabricator.wikimedia.org/P27409 and previous config saved to /var/cache/conftool/dbconfig/20220504-071035-ladsgroup.json [production]
07:10 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1136.eqiad.wmnet with reason: Maintenance [production]
07:10 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1136.eqiad.wmnet with reason: Maintenance [production]
07:10 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1098:3317 (T307525)', diff saved to https://phabricator.wikimedia.org/P27408 and previous config saved to /var/cache/conftool/dbconfig/20220504-071027-ladsgroup.json [production]
07:09 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1130 (T307525)', diff saved to https://phabricator.wikimedia.org/P27407 and previous config saved to /var/cache/conftool/dbconfig/20220504-070920-ladsgroup.json [production]
07:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1130 (T307525)', diff saved to https://phabricator.wikimedia.org/P27406 and previous config saved to /var/cache/conftool/dbconfig/20220504-070647-ladsgroup.json [production]
07:06 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1130.eqiad.wmnet with reason: Maintenance [production]