551-600 of 10000 results (23ms)
2020-10-01 ยง
20:47 <twentyafterfour@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.36.0-wmf.11 refs T263177 [production]
20:19 <twentyafterfour@deploy1001> rebuilt and synchronized wikiversions files: group0 wikis to 1.36.0-wmf.11 [production]
20:08 <twentyafterfour@deploy1001> Synchronized php-1.36.0-wmf.11/includes/parser/: sync ParserCache patches to unblock the train T264257 T263177 (duration: 00m 59s) [production]
18:40 <ebernhardson@deploy1001> Synchronized wmf-config/InitialiseSettings.php: cirrus: increase more_like recommendation cache from one to three days T264053 (duration: 00m 59s) [production]
17:49 <fdans@deploy1001> Finished deploy [analytics/refinery@530b339]: Regular analytics weekly train 530b339 (duration: 13m 42s) [production]
17:35 <fdans@deploy1001> Started deploy [analytics/refinery@530b339]: Regular analytics weekly train 530b339 [production]
17:26 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
17:24 <fdans@deploy1001> Finished deploy [analytics/refinery@530b339]: Regular analytics weekly train 530b339 (duration: 01m 34s) [production]
17:24 <mutante> etherpad1002 - attempted to upgrade Etherpad to newer version but wasn't working, reverted to previous one [production]
17:22 <fdans@deploy1001> Started deploy [analytics/refinery@530b339]: Regular analytics weekly train 530b339 [production]
17:16 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
16:46 <volans> migrating esams DNS records to the autogenerated ones from Netbox - T258729 [production]
16:19 <bblack> rebooting lvs1016 to a fresh state for interface config and error counters, etc - T264227 [production]
15:56 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:54 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:53 <bblack> lvs1016: re-disabled puppet with ticket ref in comment, downed interface enp5s0f0 since it's flapping furiously - T264227 [production]
15:53 <bblack> lvs1016: re-disabled puppet with ticket ref in comment, downed interface enp5s0f0 since it's flapping furiously [production]
14:55 <jayme> running ipvsadm -D -t 10.2.2.10:8081; ipvsadm -D -t 10.2.2.47:8889 on lvs1015.eqiad.wmnet - T244843 T255878 [production]
14:55 <moritzm> installing npm security updates on buster [production]
14:54 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:53 <jayme> running ipvsadm -D -t 10.2.1.10:8081; ipvsadm -D -t 10.2.1.47:8889 on lvs2010.codfw.wmnet,lvs2009.codfw.wmnet - T244843 T255878 [production]
14:52 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:50 <jayme> restarting pybal on lvs1015.eqiad.wmnet,lvs2009.codfw.wmnet - T244843 T255878 [production]
14:48 <jayme> restarting pybal on lvs2010.codfw.wmnet - T244843 T255878 [production]
14:42 <jayme> running puppet on lvs servers - T244843 T255878 [production]
14:35 <Urbanecm> Create bot_passwords table at all private wikis (T258356) [production]
14:29 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:29 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:29 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:29 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:29 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:29 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:21 <kormat@cumin1001> dbctl commit (dc=all): 'db2136 (re)pooling @ 100%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12886 and previous config saved to /var/cache/conftool/dbconfig/20201001-142156-kormat.json [production]
14:14 <andrewbogott> reimaging cloudvirt-wdqs1001 to buster [production]
14:12 <effie> enable puppet on mw2271 [production]
14:08 <moritzm> installing pillow security updates [production]
14:06 <kormat@cumin1001> dbctl commit (dc=all): 'db2136 (re)pooling @ 67%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12885 and previous config saved to /var/cache/conftool/dbconfig/20201001-140653-kormat.json [production]
13:59 <moritzm> installing nginx security updates on schema* [production]
13:51 <kormat@cumin1001> dbctl commit (dc=all): 'db2136 (re)pooling @ 33%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12884 and previous config saved to /var/cache/conftool/dbconfig/20201001-135149-kormat.json [production]
13:50 <klausman> rebooting an-worker1096 for cluster maintenance [production]
13:49 <klausman@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:49 <klausman@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:43 <vgutierrez> use synthetic warning for 2% of ECDHE-ECDSA-AES128-SHA pageviews - T258405 [production]
13:29 <moritzm> restarting mw canaries to pick up curl update [production]
13:22 <moritzm> installing curl security updates on stretch [production]
12:57 <kormat@cumin1001> dbctl commit (dc=all): 'db2136 depooling: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12883 and previous config saved to /var/cache/conftool/dbconfig/20201001-125707-kormat.json [production]
12:56 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
12:56 <kormat@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:39 <kormat@cumin1001> dbctl commit (dc=all): 'db2119 (re)pooling @ 100%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12882 and previous config saved to /var/cache/conftool/dbconfig/20201001-123925-kormat.json [production]
12:24 <kormat@cumin1001> dbctl commit (dc=all): 'db2119 (re)pooling @ 75%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12881 and previous config saved to /var/cache/conftool/dbconfig/20201001-122422-kormat.json [production]