3151-3200 of 10000 results (19ms)
2020-11-25 §
06:38 <marostegui> Stop mysql on db1125:3317 to clone clouddb1014:3317 clouddb1018:3317 T267090 [production]
06:33 <marostegui> Restart clouddb1019:3314, clouddb1019:3316 [production]
06:32 <marostegui> Restart clouddb1015:3314, clouddb1015:3316 [production]
06:28 <marostegui> Check private data on clouddb1014:3312 and clouddb1018:3312 T267090 [production]
05:48 <marostegui> Sanitize clouddb1014:3312 and clouddb1018:3312 T267090 [production]
01:10 <tgr_> Evening deploys done [production]
01:07 <tgr@deploy1001> Finished scap: Backport: [[gerrit:643156|GrowthExperiments: Add Russian aliases (T268519)]] (duration: 32m 09s) [production]
00:35 <tgr@deploy1001> Started scap: Backport: [[gerrit:643156|GrowthExperiments: Add Russian aliases (T268519)]] [production]
2020-11-24 §
23:50 <crusnov@deploy1001> Finished deploy [netbox/deploy@0362a12]: Test deploy of 2.9.10 to netbox-next T266488 p2 (duration: 00m 05s) [production]
23:50 <crusnov@deploy1001> Started deploy [netbox/deploy@0362a12]: Test deploy of 2.9.10 to netbox-next T266488 p2 [production]
23:50 <crusnov@deploy1001> Finished deploy [netbox/deploy@0362a12]: Test deploy of 2.9.10 to netbox-next T266488 (duration: 01m 51s) [production]
23:48 <crusnov@deploy1001> Started deploy [netbox/deploy@0362a12]: Test deploy of 2.9.10 to netbox-next T266488 [production]
21:27 <andrewbogott> restarting slapd on serpens [production]
21:20 <cdanis> ✔️ cdanis@seaborgium.wikimedia.org ~ 🕟🍵 sudo systemctl restart prometheus-openldap-exporter.service [production]
21:17 <andrewbogott> restarting slapd on seaborgium [production]
20:49 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
20:42 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
20:41 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
20:40 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
19:53 <otto@deploy1001> Synchronized wmf-config/InitialiseSettings-labs.php: Remove no longer needed EventLoggingSchemas override for NavigationTiming and ResourceTiming - T254606 (duration: 01m 01s) [production]
19:49 <ryankemper> [elasticsearch] Restarted all elasticsearch systemd-managed services on `relforge100[1,2]`: `elasticsearch_6@relforge-eqiad.service` and `elasticsearch_6@relforge-eqiad-small-alpha.service` [production]
19:30 <gilles@deploy1001> Synchronized php-1.36.0-wmf.18/extensions/NavigationTiming/extension.json: (no justification provided) (duration: 00m 57s) [production]
19:24 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 331a129: Remove temporary feature flags (T258116) (duration: 00m 57s) [production]
19:20 <mutante> LDAP - added derick to group nda (T268150) [production]
19:17 <moritzm> installing Java security updates on elastic* and relforge* [production]
19:09 <ppchelko@deploy1001> Synchronized wmf-config/InitialiseSettings.php: gerrit:643260 group1: Switch ParserCache to JSON (duration: 00m 57s) [production]
18:59 <mbsantos@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'push-notifications' for release 'main' . [production]
18:56 <elukey@deploy1001> Finished deploy [analytics/refinery@1ff0868]: Regular analytics weekly train (duration: 09m 50s) [production]
18:56 <volans> migrating anycast zonefile to the Netbox-generated ones - T258729 [production]
18:55 <mbsantos@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'push-notifications' for release 'main' . [production]
18:52 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
18:51 <mbsantos@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'push-notifications' for release 'main' . [production]
18:48 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
18:46 <elukey@deploy1001> Started deploy [analytics/refinery@1ff0868]: Regular analytics weekly train [production]
18:45 <crusnov@deploy1001> Finished deploy [netbox/deploy@88f61d0]: Test deploy of 2.9.9 to netbox-next T266488 p2 (duration: 00m 05s) [production]
18:45 <crusnov@deploy1001> Started deploy [netbox/deploy@88f61d0]: Test deploy of 2.9.9 to netbox-next T266488 p2 [production]
18:45 <crusnov@deploy1001> Finished deploy [netbox/deploy@88f61d0]: Test deploy of 2.9.9 to netbox-next T266488 (duration: 01m 09s) [production]
18:45 <elukey> restart memcached on mw2339 to pick up the correct port (was bound on 11211 rather than 11210) [production]
18:44 <crusnov@deploy1001> Started deploy [netbox/deploy@88f61d0]: Test deploy of 2.9.9 to netbox-next T266488 [production]
18:19 <ejegg> updated Fundraising CiviCRM from 28464df973 to fb0ad7f39b [production]
18:07 <mbsantos@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . [production]
18:06 <mbsantos@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . [production]
18:04 <mbsantos@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'wikifeeds' for release 'staging' . [production]
17:51 <volans@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
17:44 <volans@cumin1001> START - Cookbook sre.dns.netbox [production]
17:10 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
17:08 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
17:08 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
17:07 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:29 <elukey> move analytics1064 from C2 to C3 eqiad - T267065 [production]