|
2018-04-26
§
|
| 15:18 |
<mutante> |
added LDAP user tschumann to "nda" group (T192549) |
[production] |
| 14:53 |
<ppchelko@tin> |
Finished deploy [changeprop/deploy@f2f7a84]: Commit offsets for non matched messages from time to time. (duration: 01m 26s) |
[production] |
| 14:51 |
<ppchelko@tin> |
Started deploy [changeprop/deploy@f2f7a84]: Commit offsets for non matched messages from time to time. |
[production] |
| 14:26 |
<anomie> |
Running populateRevisionLength.php on group 1 for T192189 |
[production] |
| 14:25 |
<jynus> |
stop db1069 for cloning it away |
[production] |
| 13:58 |
<marostegui> |
Compress enwiki on db1116:3311 - T190704 |
[production] |
| 13:55 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1069, repool db1086 (duration: 01m 16s) |
[production] |
| 13:35 |
<zeljkof> |
EU SWAT finished |
[production] |
| 13:31 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:429178|Change chapcomwikis logo, add HD logo for chapcomwiki (T193024)]] (duration: 01m 16s) |
[production] |
| 13:30 |
<zfilipin@tin> |
Synchronized static/images/project-logos/: SWAT: [[gerrit:429178|Change chapcomwikis logo, add HD logo for chapcomwiki (T193024)]] (duration: 01m 16s) |
[production] |
| 13:19 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:428952|Add all Hindi projects plus meta as import sources for hiwikimedia (T188366)]] (duration: 01m 17s) |
[production] |
| 13:09 |
<zfilipin@tin> |
Synchronized static/images/project-logos/: SWAT: [[gerrit:428953|Fix pixelization of new wiki logos (T193028)]] (duration: 01m 17s) |
[production] |
| 12:53 |
<marostegui> |
Deploy schema change on db1113:3312 - T191519 T188299 T190148 |
[production] |
| 12:53 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1113 for alter table (duration: 01m 33s) |
[production] |
| 12:51 |
<gehel> |
reindexing lost updates on elasticsearch - T193112 |
[production] |
| 12:04 |
<mobrovac@tin> |
Finished deploy [cpjobqueue/deploy@7fbb152]: Support the exclude_topics config stanza (duration: 01m 12s) |
[production] |
| 12:03 |
<mobrovac@tin> |
Started deploy [cpjobqueue/deploy@7fbb152]: Support the exclude_topics config stanza |
[production] |
| 10:35 |
<moritzm> |
reimaging mw1312 mw1317, mw1339 (API servers) to stretch |
[production] |
| 10:29 |
<moritzm> |
reimaging mw1269, mw1323, mw1324 (app servers) to stretch |
[production] |
| 09:57 |
<marostegui> |
Drop prefswitch_survey on s1 - T173439 |
[production] |
| 09:50 |
<godog> |
eqiad-prod: more weight to ms-be104[0-3] for container/account - T190081 |
[production] |
| 09:45 |
<marostegui> |
Drop prefswitch_survey on s3 - T173439 |
[production] |
| 09:41 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1109 with low load (duration: 01m 16s) |
[production] |
| 09:30 |
<marostegui> |
Drop prefswitch_survey on s7 - T173439 |
[production] |
| 09:16 |
<marostegui> |
Drop prefswitch_survey on s2 - T173439 |
[production] |
| 09:15 |
<mark> |
Temp disabling cr1-ulsfo:xe-1/2/0 (Chicago transport) due to stability issues |
[production] |
| 09:13 |
<marostegui> |
Drop prefswitch_survey on s4 - T173439 |
[production] |
| 09:02 |
<marostegui> |
Drop prefswitch_survey on s5 and s6 - T173439 |
[production] |
| 09:01 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1086 (duration: 01m 16s) |
[production] |
| 08:51 |
<moritzm> |
reimaging mw1320, mw1321, mw1322 (app servers) to stretch |
[production] |
| 08:32 |
<moritzm> |
re-attempt reimage of mw1246 (failed yesterday with an error on the puppetmaster, testing whether this can be reproduced) |
[production] |
| 08:24 |
<jynus> |
stop and upgrade db1109 |
[production] |
| 07:58 |
<marostegui> |
Deploy schema change on db1090 - T191519 T188299 T190148 |
[production] |
| 07:45 |
<jynus> |
stopping db1090 mariadb instance to move its path, port and socket |
[production] |
| 07:21 |
<gehel> |
restarting redis masters in codfw - T193112 |
[production] |
| 07:20 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1090, pool db1122 with full weight (duration: 01m 23s) |
[production] |
| 07:16 |
<gehel> |
re-enabling puppet on rdb2* - T193112 |
[production] |
| 06:19 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: dc=codfw,cluster=elasticsearch |
[production] |
| 05:18 |
<marostegui> |
Deploy schema change on dbstore1002:s2 - T191519 T188299 T190148 |
[production] |
| 04:39 |
<ebernhardson> |
unfreeze writes to elasticsearch codfw cluster |
[production] |
| 03:54 |
<_joe_> |
stopping redis replication from eqiad to codfw for the jobqueue cluster, we have an issue ongoing with CirrusSearch jobs and replication is broken |
[production] |
| 03:41 |
<ejegg> |
re-enabled ingenico recurring charge job |
[production] |
| 02:05 |
<mutante> |
mw2163 through mw2166: since the wmf-auto-reimage failed after OS but before puppet run due to "Failed to puppet_generate_certs" i manually logged in with install-console and signed puppet certs (T174431) |
[production] |