1351-1400 of 10000 results (28ms)
2021-06-02 ยง
22:19 <Amir1> setting charset of all tables in wikitech to binary (T284108 T269348) [production]
22:11 <ryankemper> T280382 `sudo -i wmf-auto-reimage-host -p T280382 --new wdqs1003.eqiad.wmnet` on `ryankemper@cumin1001` tmux session `wdqs_reimage_2` [production]
22:11 <ryankemper> T280382 `sudo -i wmf-auto-reimage-host -p T280382 wdqs1003.eqiad.wmnet` on `ryankemper@cumin1001` tmux session `wdqs_reimage_2` [production]
22:08 <ryankemper@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
22:07 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer [production]
22:07 <ryankemper@puppetmaster1001> conftool action : set/pooled=yes; selector: name=wdqs1004.eqiad.wmnet [production]
22:07 <ryankemper@puppetmaster1001> conftool action : set/pooled=yes; selector: name=wdqs2007.codfw.wmnet [production]
22:05 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
22:01 <ryankemper@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
21:59 <ryankemper> T280382 `sudo -i cookbook sre.wdqs.data-transfer --source wdqs2008.codfw.wmnet --dest wdqs2007.codfw.wmnet --reason "transferring fresh categories journal following reimage" --blazegraph_instance categories` on `ryankemper@cumin2002` tmux session `wdqs_reimage` [production]
21:59 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer [production]
21:56 <ryankemper> T280382 `sudo -i cookbook sre.wdqs.data-transfer --source wdqs1006.eqiad.wmnet --dest wdqs1004.eqiad.wmnet --reason "transferring fresh categories journal following reimage" --blazegraph_instance categories` on `ryankemper@cumin1001` tmux session `wdqs_reimage` [production]
21:55 <ryankemper@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
21:39 <ryankemper@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wdqs1004.eqiad.wmnet with reason: REIMAGE [production]
21:38 <dzahn@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host doh3002.wikimedia.org [production]
21:37 <ryankemper@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on wdqs1004.eqiad.wmnet with reason: REIMAGE [production]
21:32 <ryankemper@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wdqs2007.codfw.wmnet with reason: REIMAGE [production]
21:30 <ryankemper@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on wdqs2007.codfw.wmnet with reason: REIMAGE [production]
21:28 <dzahn@cumin1001> START - Cookbook sre.ganeti.makevm for new host doh3002.wikimedia.org [production]
21:21 <dzahn@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host doh3001.wikimedia.org [production]
21:19 <ryankemper@puppetmaster1001> conftool action : set/pooled=inactive; selector: name=wdqs2007.codfw.wmnet [production]
21:17 <ryankemper> `ryankemper@wdqs1013:~$ sudo depool` (catching up on 17.9h lag) [production]
21:12 <dzahn@cumin1001> START - Cookbook sre.ganeti.makevm for new host doh3001.wikimedia.org [production]
21:10 <ryankemper> T280382 T281437 `sudo -i wmf-auto-reimage-host -p T280382 --new wdqs2007.codfw.wmnet` on `ryankemper@cumin2002` tmux session `wdqs_reimage` [production]
21:10 <ryankemper> T280382 `sudo -i wmf-auto-reimage-host -p T280382 --new wdqs1004.eqiad.wmnet` on `ryankemper@cumin1001` tmux session `reimage` [production]
20:58 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts doh3001.wikimedia.org [production]
20:49 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission for hosts doh3001.wikimedia.org [production]
20:27 <dzahn@cumin1001> END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host doh3002.wikimedia.org [production]
20:21 <dzahn@cumin1001> START - Cookbook sre.ganeti.makevm for new host doh3002.wikimedia.org [production]
20:00 <dzahn@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host doh3001.wikimedia.org [production]
19:42 <dzahn@cumin1001> START - Cookbook sre.ganeti.makevm for new host doh3001.wikimedia.org [production]
18:37 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: e9c981d5173b1d611458f6c70b34d73476b7bbde: Revert "enwiktionary: Raise AF emergency disable treshold+count" (T283460) (duration: 00m 58s) [production]
18:11 <urbanecm> Deployed security patch for T281972 [production]
18:05 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 4bf76fc09bc06f76ce842d42b77fe6b036943b69: Make DiscussionTools replytool available for everyone on wikitech (T283119) (duration: 00m 58s) [production]
17:33 <legoktm> disabled Kadirselcuk gerrit account, +1 spam (and blocked elsewhere) [production]
16:55 <legoktm> restarted apache2 on lists1001 for https://gerrit.wikimedia.org/r/697805 [production]
16:23 <pt1979@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
16:19 <pt1979@cumin2002> START - Cookbook sre.dns.netbox [production]
16:10 <sukhe@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts cescout1001.eqiad.wmnet [production]
16:01 <pt1979@cumin2002> START - Cookbook sre.dns.netbox [production]
15:59 <sukhe@cumin1001> START - Cookbook sre.hosts.decommission for hosts cescout1001.eqiad.wmnet [production]
13:16 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1125.eqiad.wmnet with reason: REIMAGE [production]
13:14 <kormat@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1125.eqiad.wmnet with reason: REIMAGE [production]
12:05 <jbond> enable puppet fleet wide. post changing puppetdb to use nginx-light #T164456 [production]
11:54 <jbond> disable puppet fleet wide. changing puppetdb to use nginx-light #T164456 [production]
11:27 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.7/includes/actions/InfoAction.php: 85feaa15d9bbda130541adb6302f31c4372e6519: InfoAction: Cast wgNamespaceProtection to array (T283751) (duration: 01m 00s) [production]
11:08 <jbond> update mod_auth_cas T264605 [production]
11:06 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: f12e368481b6836eefa070ad5dcf52af3f39d479: Investigate MediaSearch usability on other wikis (T278984) (duration: 00m 57s) [production]
11:04 <jbond> upload libapache2-mod-auth-cas_1.2-1 for buster and stretch - #T264605 [production]
11:01 <jbond> upload libapache2-mod-auth-cas_1.2-1+wmf11u1_amd64.deb - #T264605 [production]