3051-3100 of 10000 results (87ms)
2023-07-28 §
09:07 <btullis@cumin1001> START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on db1108.eqiad.wmnet with reason: db1108 has been replaced with db1208 - leaving for a few days before decom [production]
00:20 <tgr@deploy1002> Finished scap: Backport for [[gerrit:942468|help: Fix navigation in the help panel (T342927)]] (duration: 10m 09s) [production]
00:14 <tgr@deploy1002> tgr: Continuing with sync [production]
00:11 <tgr@deploy1002> tgr: Backport for [[gerrit:942468|help: Fix navigation in the help panel (T342927)]] synced to the testservers mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option) [production]
00:10 <tgr@deploy1002> Started scap: Backport for [[gerrit:942468|help: Fix navigation in the help panel (T342927)]] [production]
2023-07-27 §
21:43 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1005.eqiad.wmnet with reason: Maintenance [production]
21:43 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on dbstore1005.eqiad.wmnet with reason: Maintenance [production]
21:43 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1224 (T342617)', diff saved to https://phabricator.wikimedia.org/P49790 and previous config saved to /var/cache/conftool/dbconfig/20230727-214302-ladsgroup.json [production]
21:27 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1224', diff saved to https://phabricator.wikimedia.org/P49789 and previous config saved to /var/cache/conftool/dbconfig/20230727-212756-ladsgroup.json [production]
21:12 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1224', diff saved to https://phabricator.wikimedia.org/P49788 and previous config saved to /var/cache/conftool/dbconfig/20230727-211250-ladsgroup.json [production]
20:57 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1224 (T342617)', diff saved to https://phabricator.wikimedia.org/P49787 and previous config saved to /var/cache/conftool/dbconfig/20230727-205744-ladsgroup.json [production]
20:34 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1224 (T342617)', diff saved to https://phabricator.wikimedia.org/P49786 and previous config saved to /var/cache/conftool/dbconfig/20230727-203435-ladsgroup.json [production]
20:34 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1224.eqiad.wmnet with reason: Maintenance [production]
20:34 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1224.eqiad.wmnet with reason: Maintenance [production]
20:34 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1213:3316 (T342617)', diff saved to https://phabricator.wikimedia.org/P49785 and previous config saved to /var/cache/conftool/dbconfig/20230727-203415-ladsgroup.json [production]
20:19 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1213:3316', diff saved to https://phabricator.wikimedia.org/P49784 and previous config saved to /var/cache/conftool/dbconfig/20230727-201908-ladsgroup.json [production]
20:04 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1213:3316', diff saved to https://phabricator.wikimedia.org/P49783 and previous config saved to /var/cache/conftool/dbconfig/20230727-200402-ladsgroup.json [production]
19:48 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1213:3316 (T342617)', diff saved to https://phabricator.wikimedia.org/P49782 and previous config saved to /var/cache/conftool/dbconfig/20230727-194856-ladsgroup.json [production]
19:35 <jhancock@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host rdb1014.eqiad.wmnet with OS bullseye [production]
19:35 <jhancock@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002" [production]
19:33 <jhancock@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002" [production]
19:18 <jhancock@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on rdb1014.eqiad.wmnet with reason: host reimage [production]
19:15 <jhancock@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on rdb1014.eqiad.wmnet with reason: host reimage [production]
19:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1213:3316 (T342617)', diff saved to https://phabricator.wikimedia.org/P49781 and previous config saved to /var/cache/conftool/dbconfig/20230727-190637-ladsgroup.json [production]
19:06 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1213.eqiad.wmnet with reason: Maintenance [production]
19:06 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1213.eqiad.wmnet with reason: Maintenance [production]
19:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1201 (T342617)', diff saved to https://phabricator.wikimedia.org/P49780 and previous config saved to /var/cache/conftool/dbconfig/20230727-190617-ladsgroup.json [production]
18:51 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1201', diff saved to https://phabricator.wikimedia.org/P49779 and previous config saved to /var/cache/conftool/dbconfig/20230727-185110-ladsgroup.json [production]
18:41 <milimetric@deploy1002> Finished deploy [analytics/refinery@1af57de] (thin): Deploying to sync script updates and static files (duration: 00m 04s) [production]
18:41 <milimetric@deploy1002> Started deploy [analytics/refinery@1af57de] (thin): Deploying to sync script updates and static files [production]
18:41 <milimetric@deploy1002> Finished deploy [analytics/refinery@1af57de]: Deploying to sync script updates and static files (duration: 08m 25s) [production]
18:36 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1201', diff saved to https://phabricator.wikimedia.org/P49778 and previous config saved to /var/cache/conftool/dbconfig/20230727-183604-ladsgroup.json [production]
18:33 <milimetric@deploy1002> Started deploy [analytics/refinery@1af57de]: Deploying to sync script updates and static files [production]
18:20 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1201 (T342617)', diff saved to https://phabricator.wikimedia.org/P49777 and previous config saved to /var/cache/conftool/dbconfig/20230727-182058-ladsgroup.json [production]
18:13 <jhancock@cumin2002> START - Cookbook sre.hosts.reimage for host rdb1014.eqiad.wmnet with OS bullseye [production]
18:12 <krinkle@deploy1002> Finished deploy [performance/navtiming@c868e79]: Rename FID labels (Ibab7118be69e50bf), Remove QuickSurveys (T336169), Add Vietnam (T340714) (duration: 00m 05s) [production]
18:12 <krinkle@deploy1002> Started deploy [performance/navtiming@c868e79]: Rename FID labels (Ibab7118be69e50bf), Remove QuickSurveys (T336169), Add Vietnam (T340714) [production]
18:00 <jhancock@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host rdb1013.eqiad.wmnet with OS bullseye [production]
18:00 <jhancock@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002" [production]
17:59 <jhancock@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002" [production]
17:56 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1201 (T342617)', diff saved to https://phabricator.wikimedia.org/P49776 and previous config saved to /var/cache/conftool/dbconfig/20230727-175659-ladsgroup.json [production]
17:56 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1201.eqiad.wmnet with reason: Maintenance [production]
17:56 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1201.eqiad.wmnet with reason: Maintenance [production]
17:56 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1187 (T342617)', diff saved to https://phabricator.wikimedia.org/P49775 and previous config saved to /var/cache/conftool/dbconfig/20230727-175638-ladsgroup.json [production]
17:44 <jhancock@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on rdb1013.eqiad.wmnet with reason: host reimage [production]
17:41 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1187', diff saved to https://phabricator.wikimedia.org/P49774 and previous config saved to /var/cache/conftool/dbconfig/20230727-174132-ladsgroup.json [production]
17:41 <jhancock@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on rdb1013.eqiad.wmnet with reason: host reimage [production]
17:26 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1187', diff saved to https://phabricator.wikimedia.org/P49773 and previous config saved to /var/cache/conftool/dbconfig/20230727-172626-ladsgroup.json [production]
17:25 <jhancock@cumin2002> START - Cookbook sre.hosts.reimage for host rdb1013.eqiad.wmnet with OS bullseye [production]
17:04 <bd808@deploy1002> helmfile [eqiad] DONE helmfile.d/services/developer-portal: apply [production]