1801-1850 of 10000 results (79ms)
2024-01-21 §
00:11 <brennen@deploy2002> Finished deploy [phabricator/deployment@24a2a2a]: initial deploy to re-imaged phab1004 (duration: 00m 13s) [production]
00:11 <brennen@deploy2002> Started deploy [phabricator/deployment@24a2a2a]: initial deploy to re-imaged phab1004 [production]
00:03 <mutante> phab1004:/usr/bin# ln -s /var/lib/scap/scap/bin/scap . [production]
00:00 <brennen@deploy2002> Installation of scap version "latest" completed for 1 hosts [production]
00:00 <brennen@deploy2002> Installing scap version "latest" for 1 hosts [production]
2024-01-20 §
23:58 <mutante> phab1004 - chown -R scap:scap /var/lib/scap [production]
23:10 <brennen@deploy2002> Installing scap version "latest" for 1 hosts [production]
22:45 <brennen@deploy2002> Finished deploy [phabricator/deployment@24a2a2a]: initial deploy to re-imaged phab1004 (duration: 00m 10s) [production]
22:44 <brennen@deploy2002> Started deploy [phabricator/deployment@24a2a2a]: initial deploy to re-imaged phab1004 [production]
22:39 <brennen@deploy2002> Finished deploy [phabricator/deployment@24a2a2a]: initial deploy to re-imaged phab1004 (duration: 00m 10s) [production]
22:39 <brennen@deploy2002> Started deploy [phabricator/deployment@24a2a2a]: initial deploy to re-imaged phab1004 [production]
22:34 <dzahn@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:30:00 on phab2002.codfw.wmnet with reason: deployment [production]
22:34 <dzahn@cumin1002> START - Cookbook sre.hosts.downtime for 0:30:00 on phab2002.codfw.wmnet with reason: deployment [production]
22:28 <brennen@deploy2002> Finished deploy [phabricator/deployment@24a2a2a]: deploy to phab2002 to pick up db config revert (part 2) (duration: 00m 54s) [production]
22:27 <brennen@deploy2002> Started deploy [phabricator/deployment@24a2a2a]: deploy to phab2002 to pick up db config revert (part 2) [production]
22:23 <brennen@deploy2002> Finished deploy [phabricator/deployment@24a2a2a]: deploy to phab2002 to pick up db config revert (duration: 00m 55s) [production]
22:22 <brennen@deploy2002> Started deploy [phabricator/deployment@24a2a2a]: deploy to phab2002 to pick up db config revert [production]
22:02 <dzahn@cumin1002> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on phabricator.wikimedia.org with reason: OS upgrade [production]
22:02 <dzahn@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on phabricator.wikimedia.org with reason: OS upgrade [production]
22:02 <dzahn@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on phab.wmfusercontent.org with reason: OS upgrade [production]
22:02 <dzahn@cumin1002> START - Cookbook sre.hosts.downtime for 1:00:00 on phab.wmfusercontent.org with reason: OS upgrade [production]
22:02 <dzahn@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host phab1004.eqiad.wmnet with OS bullseye [production]
22:02 <dzahn@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on phab1004.eqiad.wmnet with reason: OS upgrade [production]
22:01 <dzahn@cumin1002> START - Cookbook sre.hosts.downtime for 1:00:00 on phab1004.eqiad.wmnet with reason: OS upgrade [production]
21:46 <dzahn@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on phab1004.eqiad.wmnet with reason: host reimage [production]
21:43 <dzahn@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on phab1004.eqiad.wmnet with reason: host reimage [production]
21:33 <dzahn@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on phab2002.codfw.wmnet with reason: deployment [production]
21:33 <dzahn@cumin1002> START - Cookbook sre.hosts.downtime for 1:00:00 on phab2002.codfw.wmnet with reason: deployment [production]
21:31 <dzahn@cumin1002> START - Cookbook sre.hosts.reimage for host phab1004.eqiad.wmnet with OS bullseye [production]
21:27 <dzahn@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host phab1004.eqiad.wmnet with OS bullseye [production]
21:27 <dzahn@cumin1002> START - Cookbook sre.hosts.reimage for host phab1004.eqiad.wmnet with OS bullseye [production]
21:03 <brennen@deploy2002> Finished deploy [phabricator/deployment@24a2a2a]: deploy to phab2002 to pick up db config changes (redux) (duration: 01m 35s) [production]
21:02 <brennen@deploy2002> Started deploy [phabricator/deployment@24a2a2a]: deploy to phab2002 to pick up db config changes (redux) [production]
20:38 <dzahn@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on phab2002.codfw.wmnet with reason: maintenance [production]
20:38 <dzahn@cumin1002> START - Cookbook sre.hosts.downtime for 1:00:00 on phab2002.codfw.wmnet with reason: maintenance [production]
20:37 <brennen@deploy2002> Finished deploy [phabricator/deployment@24a2a2a]: deploy to phab2002 to pick up database changes (duration: 00m 53s) [production]
20:36 <brennen@deploy2002> Started deploy [phabricator/deployment@24a2a2a]: deploy to phab2002 to pick up database changes [production]
20:32 <mutante> phabricator going down for maintenance [production]
20:24 <dzahn@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on phab.wmfusercontent.org with reason: OS upgrade [production]
20:23 <dzahn@cumin1002> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on phabricator.wikimedia.org with reason: OS upgrade [production]
20:23 <dzahn@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on phabricator.wikimedia.org with reason: OS upgrade [production]
20:22 <dzahn@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on phab1004.eqiad.wmnet with reason: OS upgrade [production]
20:22 <dzahn@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on phab1004.eqiad.wmnet with reason: OS upgrade [production]
20:04 <brennen> start of phab/phorge bullseye update window - T334519 [production]
20:01 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2106 (T352010)', diff saved to https://phabricator.wikimedia.org/P55089 and previous config saved to /var/cache/conftool/dbconfig/20240120-200154-ladsgroup.json [production]
20:01 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2106.codfw.wmnet with reason: Maintenance [production]
20:01 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2106.codfw.wmnet with reason: Maintenance [production]
14:44 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2099.codfw.wmnet with reason: Maintenance [production]
14:44 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2099.codfw.wmnet with reason: Maintenance [production]
09:53 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]