1-50 of 10000 results (18ms)
2025-07-16 ยง
23:50 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2225', diff saved to https://phabricator.wikimedia.org/P79273 and previous config saved to /var/cache/conftool/dbconfig/20250716-235029-marostegui.json [production]
23:35 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2225 (T399249)', diff saved to https://phabricator.wikimedia.org/P79271 and previous config saved to /var/cache/conftool/dbconfig/20250716-233522-marostegui.json [production]
22:59 <eileen> civicrm upgraded from 71b58ed5 to bf098cc5 [production]
22:48 <vriley@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - vriley@cumin1002" [production]
22:34 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2225 (T399249)', diff saved to https://phabricator.wikimedia.org/P79270 and previous config saved to /var/cache/conftool/dbconfig/20250716-223442-marostegui.json [production]
22:34 <marostegui@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2225.codfw.wmnet with reason: Maintenance [production]
22:34 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2207 (T399249)', diff saved to https://phabricator.wikimedia.org/P79269 and previous config saved to /var/cache/conftool/dbconfig/20250716-223419-marostegui.json [production]
22:34 <vriley@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ganeti1053.eqiad.wmnet with reason: host reimage [production]
22:28 <vriley@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on ganeti1053.eqiad.wmnet with reason: host reimage [production]
22:19 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2207', diff saved to https://phabricator.wikimedia.org/P79268 and previous config saved to /var/cache/conftool/dbconfig/20250716-221912-marostegui.json [production]
22:05 <vriley@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1053.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:04 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2207', diff saved to https://phabricator.wikimedia.org/P79267 and previous config saved to /var/cache/conftool/dbconfig/20250716-220405-marostegui.json [production]
21:58 <vriley@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1053.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
21:48 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2207 (T399249)', diff saved to https://phabricator.wikimedia.org/P79266 and previous config saved to /var/cache/conftool/dbconfig/20250716-214857-marostegui.json [production]
21:38 <vriley@cumin1002> START - Cookbook sre.hosts.reimage for host ganeti1053.eqiad.wmnet with OS bookworm [production]
21:28 <vriley@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ganeti1053.eqiad.wmnet with OS bookworm [production]
21:27 <bd808> Unblocked 191.19.0.0/17 (T399720) [releng]
21:24 <bvibber@deploy1003> Finished scap sync-world: Backport for [[gerrit:1170197|API action=chartinfo internal helper for Charts stats (T393950)]], [[gerrit:1170199|API action=chartinfo internal helper for Charts stats (T393950)]] (duration: 42m 44s) [production]
21:10 <bvibber@deploy1003> bvibber: Continuing with sync [production]
21:09 <bvibber@deploy1003> bvibber: Backport for [[gerrit:1170197|API action=chartinfo internal helper for Charts stats (T393950)]], [[gerrit:1170199|API action=chartinfo internal helper for Charts stats (T393950)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
21:01 <James_F> Docker: [php83] Re-build based on Wikimedia-provided binaries, for T398491 [releng]
20:51 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2207 (T399249)', diff saved to https://phabricator.wikimedia.org/P79265 and previous config saved to /var/cache/conftool/dbconfig/20250716-205132-marostegui.json [production]
20:51 <marostegui@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2207.codfw.wmnet with reason: Maintenance [production]
20:41 <bvibber@deploy1003> Started scap sync-world: Backport for [[gerrit:1170197|API action=chartinfo internal helper for Charts stats (T393950)]], [[gerrit:1170199|API action=chartinfo internal helper for Charts stats (T393950)]] [production]
20:39 <dani@deploy1003> helmfile [codfw] DONE helmfile.d/services/miscweb: apply [production]
20:39 <dani@deploy1003> helmfile [codfw] START helmfile.d/services/miscweb: apply [production]
20:39 <dani@deploy1003> helmfile [eqiad] DONE helmfile.d/services/miscweb: apply [production]
20:39 <dani@deploy1003> helmfile [eqiad] START helmfile.d/services/miscweb: apply [production]
20:39 <dani@deploy1003> helmfile [staging] DONE helmfile.d/services/miscweb: apply [production]
20:39 <dani@deploy1003> helmfile [staging] START helmfile.d/services/miscweb: apply [production]
20:38 <dani@deploy1003> helmfile [codfw] DONE helmfile.d/services/miscweb: apply [production]
20:38 <dani@deploy1003> helmfile [codfw] START helmfile.d/services/miscweb: apply [production]
20:38 <dani@deploy1003> helmfile [eqiad] DONE helmfile.d/services/miscweb: apply [production]
20:38 <dani@deploy1003> helmfile [eqiad] START helmfile.d/services/miscweb: apply [production]
20:38 <dani@deploy1003> helmfile [staging] DONE helmfile.d/services/miscweb: apply [production]
20:38 <dani@deploy1003> helmfile [staging] START helmfile.d/services/miscweb: apply [production]
20:31 <bvibber@deploy1003> Finished scap sync-world: Backport for [[gerrit:1170202|CX Translation::getStatus: Fix method to properly return the status (T399732)]] (duration: 10m 38s) [production]
20:26 <dzahn@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on aqs1012.eqiad.wmnet with reason: T396970 [production]
20:26 <bvibber@deploy1003> bvibber, sbisson: Continuing with sync [production]
20:23 <sukhe> sukhe@cumin1003:~$ homer cr3-eqsin* commit "drain codfw transport" [production]
20:23 <bvibber@deploy1003> bvibber, sbisson: Backport for [[gerrit:1170202|CX Translation::getStatus: Fix method to properly return the status (T399732)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
20:23 <sukhe> sukhe@cumin1003:~$ homer cr1-codfw* commit "drain eqsin transport" [production]
20:22 <sukhe> drain IC-331929 Arelion eqsin->codfw [production]
20:21 <bvibber@deploy1003> Started scap sync-world: Backport for [[gerrit:1170202|CX Translation::getStatus: Fix method to properly return the status (T399732)]] [production]
20:18 <ebernhardson@deploy1003> Finished deploy [airflow-dags/search@bc4d35c]: push updated rdf-spark-tools 0.3.159 artifact (duration: 00m 20s) [production]
20:18 <bvibber@deploy1003> Finished scap sync-world: Backport for [[gerrit:1170207|cirrus: configure managed cluster list]], [[gerrit:1170206|Pre-deploy Readers Use Cases Survey v2 (T399736)]] (duration: 09m 04s) [production]
20:18 <ebernhardson@deploy1003> Started deploy [airflow-dags/search@bc4d35c]: push updated rdf-spark-tools 0.3.159 artifact [production]
20:12 <bvibber@deploy1003> ebernhardson, dani, bvibber: Continuing with sync [production]
20:11 <bvibber@deploy1003> ebernhardson, dani, bvibber: Backport for [[gerrit:1170207|cirrus: configure managed cluster list]], [[gerrit:1170206|Pre-deploy Readers Use Cases Survey v2 (T399736)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
20:09 <bvibber@deploy1003> Started scap sync-world: Backport for [[gerrit:1170207|cirrus: configure managed cluster list]], [[gerrit:1170206|Pre-deploy Readers Use Cases Survey v2 (T399736)]] [production]