351-400 of 10000 results (49ms)
2018-07-25 §
14:29 <jynus@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool es1014 fully (duration: 00m 55s) [production]
14:00 <zfilipin@deploy1001> Synchronized php: group1 wikis to 1.32.0-wmf.14 (duration: 00m 55s) [production]
13:59 <zfilipin@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.32.0-wmf.14 [production]
13:54 <jynus> running compare.py on all es3 databases [production]
13:48 <ema> text-eqiad: test alternate domains patch with varnish 5.1.3-1wm9 T164609 [production]
13:44 <Amir1> restarting ores celery workers on codfw [production]
13:34 <ema> repool cp1067 w/ alternate domains patch and varnish 5.1.3-1wm9 T164609 [production]
13:27 <zfilipin@deploy1001> rebuilt and synchronized wikiversions files: group0 to 1.32.0-wmf.14 [production]
13:26 <ema> depool cp1067 and test alternate domains patch with varnish 5.1.3-1wm9 T164609 [production]
13:00 <gehel> resetting postgres data on maps1002 after failing replication - T200228 [production]
12:21 <Amir1> start of ladsgroup@mwmaint1001:~$ foreachwikiindblist s5 populateChangeTagDef.php --sleep 2 (T193873) [production]
12:19 <zfilipin@deploy1001> Finished scap: testwiki to php-1.32.0-wmf.14 and rebuild l10n cache (duration: 59m 27s) [production]
11:20 <zfilipin@deploy1001> Started scap: testwiki to php-1.32.0-wmf.14 and rebuild l10n cache [production]
11:16 <dcausse> EU SWAT done [production]
11:14 <dcausse@deploy1001> Synchronized ./wmf-config/CirrusSearch-common.php: [cirrus] allow term_freq and remove deprecated settings (duration: 00m 48s) [production]
11:01 <zfilipin@deploy1001> Pruned MediaWiki: 1.32.0-wmf.10 [keeping static files] (duration: 05m 13s) [production]
10:21 <jynus@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool es1014 with low load after maintenance (duration: 00m 47s) [production]
10:07 <marostegui> Deploy schema change on db2040 (s7 codfw master) with replication, this will generate lag on s7 codfw T144010 T51190 T199368 [production]
10:05 <ema> upgrade varnish to 5.1.3-1wm9 on text-eqiad T164609 [production]
09:55 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1121 (duration: 00m 47s) [production]
09:53 <godog> bounce grafana on krypton [production]
09:31 <ema> upload varnish 5.1.3-1wm9 to apt.w.o (fixing POST requests w/ separate VCL) T164609 [production]
09:20 <gehel> restarting elasticsearch cluster restart on codfw - T156137 [production]
08:41 <jynus> stop es1014 for reimage [production]
08:14 <marostegui> Deploy schema change on db1121 with replication, this will generate lag on labsdb hosts for s4 T144010 T51190 T199368 [production]
08:11 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1121 (duration: 00m 47s) [production]
08:10 <Amir1> start of ladsgroup@mwmaint1001:~$ foreachwikiindblist s4 populateChangeTagDef.php --sleep 2 (T193873) [production]
08:05 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1097:3314 db1091 (duration: 00m 47s) [production]
08:02 <gehel> pausing elasticsearch cluster restart on codfw - T156137 [production]
07:58 <jynus@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool es1014 (duration: 00m 48s) [production]
07:35 <gehel> rolling restart of elasticsearch / cirrus / codfw to disable G1 - T156137 [production]
07:12 <marostegui> Deploy schema change on db1091 T144010 T51190 T199368 [production]
06:53 <gehel> resetting postgres data on maps1004 after failing replication - T200228 [production]
06:38 <marostegui> Stop replication in sync on db1091 and db1097:3314 [production]
06:38 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1097:3314 db1091 (duration: 00m 48s) [production]
06:33 <jynus> finished es1014 -> es1017 switch T197073 [production]
06:27 <jynus> enabling semi-sync master on es1017, disabling it as client [production]
06:21 <jynus> deploy es3-master dns change [production]
06:02 <jynus@deploy1001> Synchronized wmf-config/db-eqiad.php: Switchover es3 master eqiad from es1014 to es1017 (duration: 00m 24s) [production]
06:01 <jynus> switchover es3 eqiad master from es1014 to es1017 [production]
04:35 <tstarling@deploy1001> Synchronized php-1.32.0-wmf.13/includes/api/ApiMain.php: record all API requests in statsd (duration: 00m 49s) [production]
02:43 <l10nupdate@deploy1001> ResourceLoader cache refresh completed at Wed Jul 25 02:43:05 UTC 2018 (duration 10m 19s) [production]
02:32 <l10nupdate@deploy1001> scap sync-l10n completed (1.32.0-wmf.13) (duration: 13m 28s) [production]
2018-07-24 §
21:59 <XioNoX> re-pooling eqsin [production]
21:15 <XioNoX> re1 is master routing engine on cr1-eqsin, triggering a re switch [production]
21:10 <XioNoX> starting to see recoveries from cr1-eqsin upgrade [production]
21:06 <XioNoX> Install done, cr1-eqsin re-rebooting [production]
21:00 <XioNoX> restarting cr1-eqsin for software upgrade [production]
20:32 <XioNoX> depooling eqsin for cr1-eqsin software upgrade [production]
19:09 <gehel> resetting postgres data on maps1003 after failing replication - T200228 [production]