1-50 of 10000 results (25ms)
2021-04-07 ยง
23:38 <ejegg> updated payments-wiki from b06009c099 to 70f5163816, [production]
23:35 <cstone> civicrm revision changed from eb9379daa3 to fdb4f90c74 [production]
23:10 <urbanecm@deploy1002> Synchronized wmf-config/Wikibase.php: 321bf91da7e823f026c2c2bdcc57d8cf60a52ba5: Wikibase: sample function call counters at 1:100 (T277817) (duration: 01m 08s) [production]
22:49 <mforns@deploy1002> Finished deploy [analytics/refinery@1dbbd3d] (hadoop-test): Regular analytics weekly train TEST retry1 [analytics/refinery@1dbbd3dfa996d2e970eb1cbc0a63d53040d4e3a3] (duration: 01m 47s) [production]
22:48 <mforns@deploy1002> Started deploy [analytics/refinery@1dbbd3d] (hadoop-test): Regular analytics weekly train TEST retry1 [analytics/refinery@1dbbd3dfa996d2e970eb1cbc0a63d53040d4e3a3] [production]
22:33 <mforns@deploy1002> Finished deploy [analytics/refinery@1dbbd3d] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@1dbbd3dfa996d2e970eb1cbc0a63d53040d4e3a3] (duration: 04m 15s) [production]
22:29 <mforns@deploy1002> Started deploy [analytics/refinery@1dbbd3d] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@1dbbd3dfa996d2e970eb1cbc0a63d53040d4e3a3] [production]
22:29 <mforns@deploy1002> Finished deploy [analytics/refinery@1dbbd3d] (thin): Regular analytics weekly train THIN [analytics/refinery@1dbbd3dfa996d2e970eb1cbc0a63d53040d4e3a3] (duration: 00m 07s) [production]
22:29 <mforns@deploy1002> Started deploy [analytics/refinery@1dbbd3d] (thin): Regular analytics weekly train THIN [analytics/refinery@1dbbd3dfa996d2e970eb1cbc0a63d53040d4e3a3] [production]
22:28 <mforns@deploy1002> Finished deploy [analytics/refinery@1dbbd3d]: Regular analytics weekly train [analytics/refinery@1dbbd3dfa996d2e970eb1cbc0a63d53040d4e3a3] (duration: 42m 54s) [production]
22:03 <Amir1> clearing watchlist of bots in wikidatawiki (https://www.wikidata.org/w/index.php?title=Wikidata:Project_chat&oldid=1397670734#Clean_up_watchlist_of_bots) [production]
22:01 <legoktm> deployed patch for T279451 (part 2) [production]
21:45 <mforns@deploy1002> Started deploy [analytics/refinery@1dbbd3d]: Regular analytics weekly train [analytics/refinery@1dbbd3dfa996d2e970eb1cbc0a63d53040d4e3a3] [production]
21:31 <legoktm> deployed patch for T279451 [production]
21:22 <mutante> mw2397 through mw2402 - pooled as new API appservers after scap pull and all monitoring green (T278396) [production]
21:21 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw240[0-2].codfw.wmnet [production]
21:21 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw239[7-9].codfw.wmnet [production]
21:05 <mutante> mw2397 - mw2402 - scap pull [production]
21:04 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw240[0-2].codfw.wmnet [production]
21:04 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw239[7-9].codfw.wmnet [production]
21:04 <dzahn@cumin1001> conftool action : set/weight=30; selector: name=mw240[0-2].codfw.wmnet [production]
21:03 <Amir1> clearing watchlist of bots in enwiki (https://en.wikipedia.org/w/index.php?title=Wikipedia:Bots/Noticeboard&oldid=1016563560#Clearing_bot_watchlists) [production]
21:02 <dzahn@cumin1001> conftool action : set/weight=30; selector: name=mw239[7-9].codfw.wmnet [production]
20:58 <pt1979@cumin2001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
20:56 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on mw[2400-2401].codfw.wmnet with reason: new_install [production]
20:56 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on mw[2400-2401].codfw.wmnet with reason: new_install [production]
20:56 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on mw[2397-2399].codfw.wmnet with reason: new_install [production]
20:55 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on mw[2397-2399].codfw.wmnet with reason: new_install [production]
20:54 <pt1979@cumin2001> START - Cookbook sre.dns.netbox [production]
20:54 <mutante> mw2397 - mw2402 - rebooting [production]
20:51 <pt1979@cumin2001> END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) [production]
20:44 <pt1979@cumin2001> START - Cookbook sre.dns.netbox [production]
20:30 <mutante> mw2397 through mw2402 - new hardware moving into production, initial puppet runs as appservers, added to monitoring etc (T278396) [production]
19:47 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on conf2006.codfw.wmnet with reason: REIMAGE [production]
19:45 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime for 2:00:00 on conf2006.codfw.wmnet with reason: REIMAGE [production]
19:35 <krinkle@deploy1002> Synchronized wmf-config/CommonSettings.php: no-op for Beta (disable LocalisationUpdate extension) (duration: 01m 06s) [production]
19:24 <dduvall@deploy1002> Synchronized php: group1 wikis to 1.36.0-wmf.38 (duration: 01m 06s) [production]
19:23 <dduvall@deploy1002> rebuilt and synchronized wikiversions files: group1 wikis to 1.36.0-wmf.38 [production]
19:21 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on conf2005.codfw.wmnet with reason: REIMAGE [production]
19:19 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime for 2:00:00 on conf2005.codfw.wmnet with reason: REIMAGE [production]
18:54 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on conf2004.codfw.wmnet with reason: REIMAGE [production]
18:52 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime for 2:00:00 on conf2004.codfw.wmnet with reason: REIMAGE [production]
17:40 <tgr@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'linkrecommendation' for release 'production' . [production]
17:40 <tgr@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'linkrecommendation' for release 'external' . [production]
17:29 <tgr@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'linkrecommendation' for release 'external' . [production]
17:29 <tgr@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'linkrecommendation' for release 'production' . [production]
17:20 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on moss-fe2002.codfw.wmnet with reason: REIMAGE [production]
17:18 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime for 2:00:00 on moss-fe2002.codfw.wmnet with reason: REIMAGE [production]
16:45 <tgr@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'linkrecommendation' for release 'staging' . [production]
15:47 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on moss-fe2001.codfw.wmnet with reason: REIMAGE [production]