5751-5800 of 10000 results (73ms)
2019-08-22 §
05:06 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Remove db2059 from config T230884 (duration: 00m 59s) [production]
00:37 <XioNoX> run /usr/local/sbin/restart-php7.2-fpm on mwdebug1001/2 [production]
00:27 <XioNoX> push L3 ECMP to eqiad routers - T230955 [production]
00:23 <XioNoX> push L3 ECMP to esams routers - T230955 [production]
00:22 <XioNoX> push L3 ECMP to eqsin routers - T230955 [production]
00:21 <twentyafterfour> phabricator update completed without incident [production]
00:19 <XioNoX> push L3 ECMP to codfw routers - T230955 [production]
00:15 <twentyafterfour> Starting phabricator upgrade from tag release/2019-08-14/1 to release/2019-08-22/1 [production]
2019-08-21 §
21:44 <eileen> civicrm revision changed from d7370a9d0b to ab2a9b264b, config revision is 58cd6b7ae6 [production]
20:51 <ebernhardson@deploy1001> Finished deploy [search/mjolnir/deploy@fc270fd]: bulk_daemon: Retune popularity_score bulk sizing (duration: 03m 49s) [production]
20:48 <ebernhardson@deploy1001> Started deploy [search/mjolnir/deploy@fc270fd]: bulk_daemon: Retune popularity_score bulk sizing [production]
20:17 <ebernhardson@deploy1001> Finished deploy [search/mjolnir/deploy@67103e9]: bulk_daemon: Correct super() call (duration: 04m 19s) [production]
20:13 <ebernhardson@deploy1001> Started deploy [search/mjolnir/deploy@67103e9]: bulk_daemon: Correct super() call [production]
20:02 <ebernhardson@deploy1001> Finished deploy [search/mjolnir/deploy@556c4d0]: bulk_daemon: Track timeouts, log indices used, increase thread counts (duration: 04m 42s) [production]
20:00 <XioNoX> test l3 ECMP in ulsfo [production]
19:57 <ebernhardson@deploy1001> Started deploy [search/mjolnir/deploy@556c4d0]: bulk_daemon: Track timeouts, log indices used, increase thread counts [production]
19:54 <ebernhardson@deploy1001> Finished deploy [search/mjolnir/deploy@adff5ad]: bulk_daemon: Track timeouts, log indices used, increase thread counts (duration: 02m 34s) [production]
19:52 <ebernhardson@deploy1001> Started deploy [search/mjolnir/deploy@adff5ad]: bulk_daemon: Track timeouts, log indices used, increase thread counts [production]
19:34 <XioNoX> repool codfw and eqsin - T226422 [production]
19:31 <XioNoX> Rollback: Varnish: redirect eqsin/ulsfo text to eqiad - T226422 [production]
19:29 <ayounsi@puppetmaster1001> conftool action : set/pooled=true; selector: dnsdisc=kartotherian,name=codfw [production]
19:29 <ayounsi@puppetmaster1001> conftool action : set/pooled=true; selector: dnsdisc=swift,name=codfw [production]
19:26 <XioNoX> rollback: increase OSPF cost on cr2-codfw links - T226422 [production]
19:25 <XioNoX> rollback deactivate transit links on cr2-codfw - T226422 [production]
19:24 <XioNoX> rollback: move VRRP master from cr2-codfw to cr1-codfw - T226422 [production]
19:16 <XioNoX> restart both REs on cr2-codfw - T226422 [production]
19:14 <XioNoX> failover master RE to RE0 on cr2-codfw - T226422 [production]
18:37 <XioNoX> shutdown re0:cr2-codfw (backup) - T226422 [production]
18:32 <XioNoX> failover master RE to RE1 on cr2-codfw - T226422 [production]
18:19 <XioNoX> shutdown re1:cr2-codfw (backup) - T226422 [production]
18:18 <jforrester@deploy1001> Synchronized php-1.34.0-wmf.19/includes/specialpage/RedirectSpecialPage.php: T230932 RedirectSpecialArticle: Fix PHP notice about undefined index (duration: 00m 54s) [production]
18:17 <XioNoX> move VRRP master from cr2-codfw to cr1-codfw - T226422 [production]
18:15 <ayounsi@puppetmaster1001> conftool action : set/pooled=false; selector: dnsdisc=swift,name=codfw [production]
18:15 <ayounsi@puppetmaster1001> conftool action : set/pooled=false; selector: dnsdisc=kartotherian,name=codfw [production]
18:15 <tarrow@deploy1001> Synchronized php-1.34.0-wmf.19/extensions/Wikibase/client/: SWAT: [[gerrit:531528|Use the backwards-compatible HTML ID for the wikidata item link (T66315)]] (duration: 00m 58s) [production]
18:14 <ayounsi@puppetmaster1001> conftool action : set/pooled=false; selector: dnsdisc=swift,name=codfw [production]
18:14 <ayounsi@puppetmaster1001> conftool action : set/pooled=false; selector: dnsdisc=kartotherian,name=codfw [production]
18:12 <XioNoX> deactivate transit links on cr2-codfw - T226422 [production]
18:04 <XioNoX> increase OSPF cost on cr2-codfw links - T226422 [production]
17:56 <XioNoX> rollback: apply BGP graceful shutdown to cr1-codfw transits - T226422 [production]
17:55 <XioNoX> Rollback: increase OSPF cost on ulsfo-codfw link - T226422 [production]
17:53 <XioNoX> rollback: disable BGP from cr1-codfw to lvs2001/2/3 - T226422 [production]
17:43 <XioNoX> restart both REs on cr1-codfw - T226422 [production]
17:33 <XioNoX> failover master RE to RE0 on cr1-codfw - T226422 [production]
17:33 <cmjohnson1> cloudvirt1015 down for a new motherboard [production]
17:25 <XioNoX> shutdown RE0 on cr1-codfw - T226422 [production]
17:17 <bstorm_> reboot cloudvirt1024 to try and reset raid T230289 [production]
17:17 <XioNoX> failover master RE to RE1 on cr1-codfw - T226422 [production]
17:08 <XioNoX> disable BGP from cr1-codfw to lvs2001/2/3 - T226422 [production]
17:02 <cmjohnson1> rebooting cloudvirt1024 [production]