2651-2700 of 10000 results (55ms)
2018-04-26 §
16:02 <ppchelko@tin> Started deploy [cpjobqueue/deploy@bf34e00]: Enable all jobs for test, test2, testwikidata and mediawiki. T190327 [production]
15:37 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Add db1090 as multiinstance (duration: 01m 16s) [production]
15:36 <jynus@tin> Synchronized wmf-config/db-codfw.php: Add db1090 as multiinstance (duration: 01m 17s) [production]
15:18 <mutante> added LDAP user tschumann to "nda" group (T192549) [production]
14:53 <ppchelko@tin> Finished deploy [changeprop/deploy@f2f7a84]: Commit offsets for non matched messages from time to time. (duration: 01m 26s) [production]
14:51 <ppchelko@tin> Started deploy [changeprop/deploy@f2f7a84]: Commit offsets for non matched messages from time to time. [production]
14:26 <anomie> Running populateRevisionLength.php on group 1 for T192189 [production]
14:25 <jynus> stop db1069 for cloning it away [production]
13:58 <marostegui> Compress enwiki on db1116:3311 - T190704 [production]
13:55 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Depool db1069, repool db1086 (duration: 01m 16s) [production]
13:35 <zeljkof> EU SWAT finished [production]
13:31 <zfilipin@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:429178|Change chapcomwikis logo, add HD logo for chapcomwiki (T193024)]] (duration: 01m 16s) [production]
13:30 <zfilipin@tin> Synchronized static/images/project-logos/: SWAT: [[gerrit:429178|Change chapcomwikis logo, add HD logo for chapcomwiki (T193024)]] (duration: 01m 16s) [production]
13:19 <zfilipin@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:428952|Add all Hindi projects plus meta as import sources for hiwikimedia (T188366)]] (duration: 01m 17s) [production]
13:09 <zfilipin@tin> Synchronized static/images/project-logos/: SWAT: [[gerrit:428953|Fix pixelization of new wiki logos (T193028)]] (duration: 01m 17s) [production]
12:53 <marostegui> Deploy schema change on db1113:3312 - T191519 T188299 T190148 [production]
12:53 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1113 for alter table (duration: 01m 33s) [production]
12:51 <gehel> reindexing lost updates on elasticsearch - T193112 [production]
12:04 <mobrovac@tin> Finished deploy [cpjobqueue/deploy@7fbb152]: Support the exclude_topics config stanza (duration: 01m 12s) [production]
12:03 <mobrovac@tin> Started deploy [cpjobqueue/deploy@7fbb152]: Support the exclude_topics config stanza [production]
10:35 <moritzm> reimaging mw1312 mw1317, mw1339 (API servers) to stretch [production]
10:29 <moritzm> reimaging mw1269, mw1323, mw1324 (app servers) to stretch [production]
09:57 <marostegui> Drop prefswitch_survey on s1 - T173439 [production]
09:50 <godog> eqiad-prod: more weight to ms-be104[0-3] for container/account - T190081 [production]
09:45 <marostegui> Drop prefswitch_survey on s3 - T173439 [production]
09:41 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Repool db1109 with low load (duration: 01m 16s) [production]
09:30 <marostegui> Drop prefswitch_survey on s7 - T173439 [production]
09:16 <marostegui> Drop prefswitch_survey on s2 - T173439 [production]
09:15 <mark> Temp disabling cr1-ulsfo:xe-1/2/0 (Chicago transport) due to stability issues [production]
09:13 <marostegui> Drop prefswitch_survey on s4 - T173439 [production]
09:02 <marostegui> Drop prefswitch_survey on s5 and s6 - T173439 [production]
09:01 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Depool db1086 (duration: 01m 16s) [production]
08:51 <moritzm> reimaging mw1320, mw1321, mw1322 (app servers) to stretch [production]
08:32 <moritzm> re-attempt reimage of mw1246 (failed yesterday with an error on the puppetmaster, testing whether this can be reproduced) [production]
08:24 <jynus> stop and upgrade db1109 [production]
07:58 <marostegui> Deploy schema change on db1090 - T191519 T188299 T190148 [production]
07:45 <jynus> stopping db1090 mariadb instance to move its path, port and socket [production]
07:21 <gehel> restarting redis masters in codfw - T193112 [production]
07:20 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Depool db1090, pool db1122 with full weight (duration: 01m 23s) [production]
07:16 <gehel> re-enabling puppet on rdb2* - T193112 [production]
06:19 <gehel@puppetmaster1001> conftool action : set/pooled=yes; selector: dc=codfw,cluster=elasticsearch [production]
05:18 <marostegui> Deploy schema change on dbstore1002:s2 - T191519 T188299 T190148 [production]
04:39 <ebernhardson> unfreeze writes to elasticsearch codfw cluster [production]
03:54 <_joe_> stopping redis replication from eqiad to codfw for the jobqueue cluster, we have an issue ongoing with CirrusSearch jobs and replication is broken [production]
03:41 <ejegg> re-enabled ingenico recurring charge job [production]
02:05 <mutante> mw2163 through mw2166: since the wmf-auto-reimage failed after OS but before puppet run due to "Failed to puppet_generate_certs" i manually logged in with install-console and signed puppet certs (T174431) [production]
2018-04-25 §
22:55 <samwilson@tin> Synchronized wmf-config/InitialiseSettings.php: Undeploy GlobalPreferences T184121 (duration: 01m 16s) [production]
22:21 <samwilson@tin> Synchronized wmf-config/InitialiseSettings.php: Deploy GlobalPreferences T189806 (duration: 01m 18s) [production]
21:03 <demon@tin> rebuilt and synchronized wikiversions files: group1 to wmf.1 [production]
21:01 <demon@tin> Synchronized php: symlink bump (duration: 01m 16s) [production]