351-400 of 10000 results (107ms)
2025-08-27 ยง
18:04 <jhancock@cumin1003> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host maps2011.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED [production]
17:46 <mutante> upgrading envoyproxy on doc* and etherpad* hosts T402584 [production]
17:40 <jhancock@cumin1003> START - Cookbook sre.hosts.provision for host maps2014.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED [production]
17:40 <jhancock@cumin1003> START - Cookbook sre.hosts.provision for host maps2013.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED [production]
17:40 <jhancock@cumin1003> START - Cookbook sre.hosts.provision for host maps2012.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED [production]
17:39 <jhancock@cumin1003> START - Cookbook sre.hosts.provision for host maps2011.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED [production]
17:39 <jhancock@cumin1003> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host maps2014 [production]
17:39 <jhancock@cumin1003> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host maps2013 [production]
17:39 <jhancock@cumin1003> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host maps2012 [production]
17:39 <jhancock@cumin1003> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host maps2011 [production]
17:39 <jhancock@cumin1003> START - Cookbook sre.network.configure-switch-interfaces for host maps2014 [production]
17:39 <jhancock@cumin1003> START - Cookbook sre.network.configure-switch-interfaces for host maps2013 [production]
17:39 <jhancock@cumin1003> START - Cookbook sre.network.configure-switch-interfaces for host maps2012 [production]
17:38 <jhancock@cumin1003> START - Cookbook sre.network.configure-switch-interfaces for host maps2011 [production]
17:38 <jhancock@cumin1003> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
17:38 <jhancock@cumin1003> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding maps2011-2014 to codfw - jhancock@cumin1003" [production]
17:38 <jhancock@cumin1003> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding maps2011-2014 to codfw - jhancock@cumin1003" [production]
17:35 <jhancock@cumin1003> START - Cookbook sre.dns.netbox [production]
17:21 <mutante> upgrading envoyproxy on releases* and planet* hosts T402584 [production]
17:05 <mutante> upgrading envoyproxy on aphlict* and zuul* hosts T402584 [production]
16:58 <mutante> upgrading envoyproxy on people* hosts T402584 [production]
16:49 <ladsgroup@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on dbstore1008.eqiad.wmnet with reason: Maintenance [production]
16:39 <ladsgroup@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1245.eqiad.wmnet with reason: Maintenance [production]
16:39 <ladsgroup@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1230 (T402925)', diff saved to https://phabricator.wikimedia.org/P81924 and previous config saved to /var/cache/conftool/dbconfig/20250827-163928-ladsgroup.json [production]
16:24 <ladsgroup@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1230', diff saved to https://phabricator.wikimedia.org/P81923 and previous config saved to /var/cache/conftool/dbconfig/20250827-162420-ladsgroup.json [production]
16:09 <ladsgroup@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1230', diff saved to https://phabricator.wikimedia.org/P81922 and previous config saved to /var/cache/conftool/dbconfig/20250827-160912-ladsgroup.json [production]
16:02 <dreamyjazz@deploy1003> Finished scap sync-world: Backport for [[gerrit:1182587|Temp accounts: Disable logged out editing on wikimaniawiki (T403067)]] (duration: 17m 29s) [production]
16:01 <sukhe@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host durum1002.eqiad.wmnet with OS trixie [production]
15:58 <James_F> jforrester@deploy1003:~$ foreachwikiindblist wikifunctionsclient sql /srv/mediawiki-staging/php-1.45.0-wmf.16/extensions/WikiLambda/sql/mysql/table-usage.sql # T403079 [production]
15:57 <dreamyjazz@deploy1003> dreamyjazz: Continuing with sync [production]
15:54 <sukhe> reprepro -C main include trixie-wikimedia anycast-healthchecker_0.9.8-1+wmf13u1_amd64.changes: T401832 [production]
15:54 <ladsgroup@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1230 (T402925)', diff saved to https://phabricator.wikimedia.org/P81920 and previous config saved to /var/cache/conftool/dbconfig/20250827-155405-ladsgroup.json [production]
15:51 <dreamyjazz@deploy1003> dreamyjazz: Backport for [[gerrit:1182587|Temp accounts: Disable logged out editing on wikimaniawiki (T403067)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
15:45 <swfrench-wmf> finished etcd cfssl-PKI migration in eqiad - T352245 [production]
15:45 <dreamyjazz@deploy1003> Started scap sync-world: Backport for [[gerrit:1182587|Temp accounts: Disable logged out editing on wikimaniawiki (T403067)]] [production]
15:43 <ladsgroup@cumin1003> dbctl commit (dc=all): 'Depooling db1230 (T402925)', diff saved to https://phabricator.wikimedia.org/P81918 and previous config saved to /var/cache/conftool/dbconfig/20250827-154350-ladsgroup.json [production]
15:43 <ladsgroup@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1230.eqiad.wmnet with reason: Maintenance [production]
15:43 <zabe@deploy1003> Finished scap sync-world: Backport for [[gerrit:1182595|Use cl_timestamp_id instead of cl_timestamp (T403069)]], [[gerrit:1182596|Use cl_timestamp_id instead of cl_timestamp (T403069)]] (duration: 10m 52s) [production]
15:37 <zabe@deploy1003> zabe: Continuing with sync [production]
15:36 <zabe@deploy1003> zabe: Backport for [[gerrit:1182595|Use cl_timestamp_id instead of cl_timestamp (T403069)]], [[gerrit:1182596|Use cl_timestamp_id instead of cl_timestamp (T403069)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
15:32 <zabe@deploy1003> Started scap sync-world: Backport for [[gerrit:1182595|Use cl_timestamp_id instead of cl_timestamp (T403069)]], [[gerrit:1182596|Use cl_timestamp_id instead of cl_timestamp (T403069)]] [production]
15:31 <ladsgroup@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1216.eqiad.wmnet with reason: Maintenance [production]
15:31 <ladsgroup@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1207 (T402925)', diff saved to https://phabricator.wikimedia.org/P81917 and previous config saved to /var/cache/conftool/dbconfig/20250827-153138-ladsgroup.json [production]
15:25 <sgimeno@deploy1003> helmfile [codfw] DONE helmfile.d/services/changeprop-jobqueue: apply [production]
15:24 <sgimeno@deploy1003> helmfile [codfw] START helmfile.d/services/changeprop-jobqueue: apply [production]
15:24 <sgimeno@deploy1003> helmfile [eqiad] DONE helmfile.d/services/changeprop-jobqueue: apply [production]
15:22 <sgimeno@deploy1003> helmfile [eqiad] START helmfile.d/services/changeprop-jobqueue: apply [production]
15:21 <sgimeno@deploy1003> helmfile [staging] DONE helmfile.d/services/changeprop-jobqueue: apply [production]
15:16 <ladsgroup@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1207', diff saved to https://phabricator.wikimedia.org/P81916 and previous config saved to /var/cache/conftool/dbconfig/20250827-151630-ladsgroup.json [production]
15:16 <sgimeno@deploy1003> helmfile [staging] START helmfile.d/services/changeprop-jobqueue: apply [production]