701-750 of 10000 results (19ms)
2020-09-29 ยง
22:33 <dzahn@cumin1001> START - Cookbook sre.ganeti.makevm [production]
22:26 <mutante> phab1001 - re-enabled puppet and running it [production]
22:24 <ejegg> CiviCRM rolled back from 4aa0aeccd1 to 035ad1c351 [production]
22:16 <eileen> civicrm revision changed from 035ad1c351 to 4aa0aeccd1, config revision is b9120969bf [production]
21:59 <mutante> temp. disabled puppet on phab1001 [production]
21:49 <mutante> restarted aphlict service on aphlict1001 [production]
21:47 <twentyafterfour@deploy1001> Finished scap: testwikis wikis to 1.36.0-wmf.10 (duration: 13m 45s) [production]
21:34 <twentyafterfour@deploy1001> Started scap: testwikis wikis to 1.36.0-wmf.10 [production]
21:30 <mutante> started DHCP service on install2003 again [production]
21:22 <mutante> temp stopping DHCP service on install2003 for a test [production]
21:09 <mutante> rebooting testvm5001 for install test after switching DHCP/TFTP in eqsin to new dedicated VM [production]
21:02 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
21:00 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:55 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
20:55 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:54 <cdanis@cumin1001> dbctl commit (dc=all): 'depool db2125', diff saved to https://phabricator.wikimedia.org/P12843 and previous config saved to /var/cache/conftool/dbconfig/20200929-205453-cdanis.json [production]
20:51 <mutante> DHCP server for EQSIN switched from bast5001 to install5001 (T252526) [production]
20:45 <twentyafterfour@deploy1001> Finished scap: testwikis to 1.36.0-wmf.11 refs T263177 (duration: 69m 57s) [production]
19:44 <andrewbogott> apt-get update && apt-get upgrade on wikitech-static [production]
19:40 <mutante> temp. disabling puppet on ms-fe (swift-proxy) hosts, applying puppet refactoring change carefully [production]
19:35 <twentyafterfour@deploy1001> Started scap: testwikis to 1.36.0-wmf.11 refs T263177 [production]
19:29 <twentyafterfour> Checked out mediawiki 1.36.0-wmf.11 on deploy1001 see T263177 [production]
17:30 <hnowlan> ported cassandra-tools-wmf to wikimedia-buster [production]
17:12 <jbond42> update libdbi-perl on dbmonitor1001 and helium [production]
17:02 <jbond42> re-enable puppet to post deploy puppetdb change [production]
16:57 <jbond42> disable puppet to deploy puppetdb change [production]
16:34 <chaomodus> deploying eqsin automated DNS [production]
15:51 <ppchelko@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
15:48 <hnowlan@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
15:47 <hnowlan@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
15:47 <hnowlan@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
15:39 <ppchelko@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
15:23 <pt1979@cumin2001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:15 <pt1979@cumin2001> START - Cookbook sre.dns.netbox [production]
15:10 <oblivian@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . [production]
15:02 <ppchelko@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]
15:00 <vgutierrez> restarting acme-chief on acmechief1001 [production]
14:48 <oblivian@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . [production]
14:43 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:41 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:38 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
14:34 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
14:32 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.init-hadoop-workers (exit_code=0) [production]
14:30 <elukey@cumin1001> START - Cookbook sre.hadoop.init-hadoop-workers [production]
14:30 <bblack> switching eqsin and esams public-facing unified certs to letsencrypt - https://gerrit.wikimedia.org/r/c/operations/puppet/+/630847 [production]
14:06 <moritzm> installing facter updates from Buster 10.6 point release [production]
13:57 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:57 <kormat@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:54 <jayme@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'citoid' for release 'production' . [production]
13:49 <kormat@cumin1001> dbctl commit (dc=all): 'Remove db2126 from dump/vslow T259831', diff saved to https://phabricator.wikimedia.org/P12841 and previous config saved to /var/cache/conftool/dbconfig/20200929-134926-kormat.json [production]