|
2019-02-27
§
|
| 19:49 |
<bstorm_> |
stopped slave on labsbd1004 for T193264 |
[production] |
| 19:43 |
<bstorm_> |
downtimed labsdb1004 to stop mysql for transferring data for T193264 |
[production] |
| 19:32 |
<SMalyshev> |
repooled wdqs1005, caught up |
[production] |
| 19:26 |
<herron> |
replacing kafka on logstash1004 with logstash1010 T213898 |
[production] |
| 18:56 |
<SMalyshev> |
depooled wdqs1005 to let it catch up |
[production] |
| 18:36 |
<smalyshev@deploy1001> |
Finished deploy [wdqs/wdqs@465673b]: Redeploy GUI for T217161 (duration: 10m 51s) |
[production] |
| 18:28 |
<cmjohnson1> |
powering off mw126[3-6] one at a time to move to different rack A5 T212348 |
[production] |
| 18:25 |
<smalyshev@deploy1001> |
Started deploy [wdqs/wdqs@465673b]: Redeploy GUI for T217161 |
[production] |
| 18:21 |
<cmjohnson1> |
powering off mw1262 to move to different rack A5 T212348 |
[production] |
| 18:15 |
<cmjohnson1> |
powering off mw1261 to move to different rack A5 T212348 |
[production] |
| 17:57 |
<niharika29@deploy1001> |
Synchronized php-1.33.0-wmf.19/extensions/Flow/: Make VisualEditor unwrap <section> tags T217206 (duration: 01m 00s) |
[production] |
| 17:56 |
<elukey> |
roll restart hadoop hdfs namenodes on an-master100[1,2] to pick up the new rack config of analytics1071 |
[production] |
| 17:37 |
<niharika29@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Welcome survey: add a control group to viwiki T216669 (duration: 00m 54s) |
[production] |
| 17:34 |
<niharika29@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Stop collecting data for CitaitonUsage and CitationUsagePageLoad T213969 (duration: 00m 55s) |
[production] |
| 17:22 |
<elukey> |
drain + shutdown of analytics1071 to allow its move to A5 - T212348 |
[production] |
| 17:19 |
<cmjohnson1> |
powering off wtp1030 to move to different rack A5 T212348 |
[production] |
| 17:14 |
<cmjohnson1> |
powering off wtp1029 to move to different rack A5 T212348 |
[production] |
| 17:06 |
<cmjohnson1> |
powering off wtp1029 to move to different rack A5 T212348 |
[production] |
| 17:05 |
<RoanKattouw> |
Running foreachwikiindblist dblists/echo.dblist extensions/Echo/maintenance/removeOrphanedEvents.php on mwmaint1002 |
[production] |
| 16:58 |
<hashar@deploy1001> |
Synchronized php-1.33.0-wmf.19/extensions/Score: Revert "beautify lilypond error message output" - T217241 (duration: 00m 56s) |
[production] |
| 16:49 |
<jijiki> |
Deploy LVS for eventgate-analytics - T211247 |
[production] |
| 16:26 |
<volans> |
temporarily disabled puppet on icinga[12]001 to deploy g/493171 |
[production] |
| 16:20 |
<volans> |
force-rebooting icinga1001 (to test some puppet changes) - T214760 |
[production] |
| 15:34 |
<jbond42> |
rolling openssl security updates to jessie canary servers |
[production] |
| 14:25 |
<marostegui> |
Deploy schema change on abuse_filter_log on s7 codfw - lag will be generated on codfw - T187295 |
[production] |
| 14:01 |
<marostegui> |
Change indexes on abuse_filter_log on db1089 - T187295 |
[production] |
| 14:00 |
<moritzm> |
uploaded openssl 1.0.2r to jessie-wikimedia |
[production] |
| 12:08 |
<jbond42> |
correction: rolling updates of apache on mw api servers *not* jobrunners |
[production] |
| 12:04 |
<jbond42> |
rolling updates of apache on mw jobrunners |
[production] |
| 11:38 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Fully repool db1077 after MySQL upgrade (duration: 00m 53s) |
[production] |
| 11:28 |
<godog> |
cleanup log4j from lvs eqiad / ipvsadm -D -t logstash.svc.eqiad.wmnet:4560 |
[production] |
| 11:25 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Give more traffic to db1077 after MySQL upgrade (duration: 00m 54s) |
[production] |
| 11:17 |
<godog> |
roll-restart pybal after removing logstash log4j service |
[production] |
| 10:54 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Give more traffic to db1077 after MySQL upgrade (duration: 00m 54s) |
[production] |
| 10:35 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1077 with low weight after MySQL upgrade (duration: 00m 53s) |
[production] |
| 09:55 |
<marostegui> |
Stop MySQL on db1077 for mysql upgrade - this will generate lag on labsdb:s3 |
[production] |
| 09:55 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1077 for MySQL upgrade (duration: 00m 53s) |
[production] |
| 09:50 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Fully repool db1082 (duration: 00m 54s) |
[production] |
| 09:35 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool in API db1082 after mysql upgrade (duration: 00m 53s) |
[production] |
| 09:21 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Slowly repool db1082 after mysql upgrade (duration: 00m 54s) |
[production] |
| 09:05 |
<marostegui> |
Stop MySQL on db1082 for mysql upgrade |
[production] |
| 08:41 |
<godog> |
enable mmjsonparse by default on kafka outputs - T213189 |
[production] |
| 08:40 |
<marostegui> |
Deploy schema change on db1082 - will generate lag on labsdb:s5 - T86342 |
[production] |
| 08:40 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1082 for mysql upgrade (duration: 00m 54s) |
[production] |
| 08:26 |
<marostegui> |
Retroactive log, T216444 Global rename of Дагиров Умар → Takhirgeran Umar was done by alanajjar |
[production] |
| 08:02 |
<marostegui> |
Global rename of HeavyTony → QTHCCAN by alanajjar - T217222 |
[production] |
| 07:01 |
<marostegui> |
Deploy schema change on s5 codfw master (db2052), this will generate lag on codfw - T86342 |
[production] |
| 06:30 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1076 (duration: 00m 55s) |
[production] |
| 06:12 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1076 (duration: 01m 08s) |
[production] |
| 05:05 |
<kart_> |
Finished manual run of unpublished ContentTranslation draft purge script (T216983) |
[production] |