3301-3350 of 10000 results (56ms)
2018-05-31 ยง
14:27 <addshore@tin> Synchronized wmf-config/Wikibase.php: [[gerrit:436495|Wikibase.php shift around the loading of WikibaseLexeme]] (duration: 01m 22s) [production]
14:21 <addshore@tin> Synchronized wmf-config/InitialiseSettings-labs.php: BETA ONLY [[gerrit:436500|gerrit]] (duration: 01m 21s) [production]
14:20 <addshore@tin> Synchronized wmf-config/Wikibase-labs.php: BETA ONLY [[gerrit:436500|gerrit]] (duration: 01m 21s) [production]
14:08 <ottomata> beginning restarts of Kafka main-eqiad to enable SSL port - T193778 [production]
13:41 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool sanitarium masters - T190704 (duration: 01m 21s) [production]
13:23 <hashar@tin> Synchronized php-1.32.0-wmf.6/vendor: WikibaseLexeme: Encoding problems in labels - T195359 (duration: 03m 12s) [production]
12:54 <akosiaris> reimage kubernetes200{3,4}.codfw.wmnet [production]
12:30 <marostegui> Stop replication on all sanitarium masters - T190704 [production]
11:14 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool sanitarium masters - T190704 (duration: 01m 22s) [production]
10:45 <elukey> removed Pivot from thorium (pivot.wikimedia.org now simply redirects to Turnilo) [production]
09:45 <marostegui> Stop MySQL on db2062 to clone db2092 [production]
09:45 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Depool db2062 for cloning db2092 (duration: 01m 22s) [production]
09:40 <volans> restarting Icinga, issues processing the command file [production]
09:35 <jynus> testing icinga alerting [production]
09:28 <elukey> reimage druid1006 to Debian Stretch [production]
09:11 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Repool db1082 fully (duration: 01m 21s) [production]
08:42 <gehel> power off elastic2018 - T196045 [production]
08:39 <Amir1> ladsgroup@terbium:~$ mwscript deleteAutoPatrolLogs.php --wiki=commonswiki --sleep 2 --check-old [production]
08:09 <gehel> power reset elastic2018 [production]
08:08 <akosiaris@puppetmaster1001> conftool action : set/pooled=inactive; selector: dc=codfw,service=mathoid,cluster=scb,name=scb.* [production]
08:07 <akosiaris@puppetmaster1001> conftool action : set/pooled=inactive; selector: dc=eqiad,service=mathoid,cluster=scb,name=scb.* [production]
07:54 <akosiaris> reimage kubernetes1004 without swap [production]
07:48 <ema> power-cycle elastic2018 [production]
07:41 <marostegui> Stop Replication on db2066 [production]
07:40 <akosiaris> re-enable puppet across the fleet [production]
07:30 <akosiaris> disable puppet for https://gerrit.wikimedia.org/r/#/c/436468/ merge cross fleet [production]
07:27 <marostegui> Deploy schema change on s5 codfw master (db2052) this will generate lag on codfw - T191316 T192926 T89737 T195193 [production]
07:24 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Repool db2066 after alter table (duration: 01m 22s) [production]
06:38 <marostegui> Deploy schema change on db2066 - T191316 T192926 T89737 T195193 [production]
06:38 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Depool db2066 for alter table (duration: 01m 21s) [production]
06:30 <marostegui> Deploy schema change on db2092:3315 and db2094:3315 - T191316 T192926 T89737 T195193 [production]
06:28 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Repool db2084:3315 after alter table (duration: 01m 28s) [production]
06:09 <akosiaris> reimage ganeti1003, ganeti1007 to stretch [production]
05:59 <elukey> reimage druid1005 to Debian Stretch [production]
05:51 <elukey> delete /tmp/scap_l10n_1501525840,scap_l10n_1501525840,l10nstuff,l10nstuff3 from tin to free some space in the root partition (1.9G left) [production]
05:40 <elukey> restart pdfrender on scb1001 [production]
05:25 <marostegui> Deploy schema change on db2084:3315 - T191316 T192926 T89737 T195193 [production]
05:24 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Depool db2084:3315 for alter table (duration: 01m 27s) [production]
05:13 <moritzm> installing python-crypto security updates on trusty [production]
04:19 <l10nupdate@tin> ResourceLoader cache refresh completed at Thu May 31 04:19:57 UTC 2018 (duration 14m 48s) [production]
04:18 <moritzm> rebooting wtp2001/wtp2015 for microcode updates [production]
04:05 <l10nupdate@tin> scap sync-l10n completed (1.32.0-wmf.6) (duration: 18m 15s) [production]
03:03 <l10nupdate@tin> scap sync-l10n completed (1.32.0-wmf.5) (duration: 16m 18s) [production]
02:09 <pnorman@tin> Finished deploy [tilerator/deploy@2a26f1e] (cleartables): Deploy style with fewer fonts (duration: 00m 24s) [production]
02:08 <pnorman@tin> Started deploy [tilerator/deploy@2a26f1e] (cleartables): Deploy style with fewer fonts [production]
02:02 <pnorman@tin> Finished deploy [tilerator/deploy@78448de] (cleartables): Deploy style with fewer fonts (duration: 00m 25s) [production]
02:01 <pnorman@tin> Started deploy [tilerator/deploy@78448de] (cleartables): Deploy style with fewer fonts [production]
01:39 <pnorman@tin> Finished deploy [tilerator/deploy@fad9969] (cleartables): Deploy updated stylesheet (duration: 00m 24s) [production]
01:39 <pnorman@tin> Started deploy [tilerator/deploy@fad9969] (cleartables): Deploy updated stylesheet [production]
01:38 <pnorman@tin> Finished deploy [tilerator/deploy@78d1b82] (cleartables): Deploy updated stylesheet (duration: 00m 05s) [production]