|
2017-06-02
§
|
| 09:49 |
<jynus> |
stopping db2041 to prepare it for reimage |
[production] |
| 09:18 |
<marostegui> |
Deploy alter table s3 - db1015 - T166278 |
[production] |
| 09:12 |
<marostegui> |
Deploy alter table s3 - labsdb1003 - T166278 |
[production] |
| 07:47 |
<marostegui> |
Resume alter table on db1047 enwiki.revision - T166452 |
[production] |
| 07:45 |
<moritzm> |
uploaded gerrit 2.13.8+git1-wmf4 to apt.wikimedia.org |
[production] |
| 07:43 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1059 - T166206 (duration: 00m 39s) |
[production] |
| 07:36 |
<marostegui> |
Deploy alter table on s4 - labsdb1009 - T166206 |
[production] |
| 07:02 |
<akosiaris> |
starting fleet wide PCC for gerrit change 356030. Should take a while to complete |
[production] |
| 05:25 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Emergency pool of db1049 (duration: 00m 48s) |
[production] |
| 04:42 |
<elukey> |
removed some old scap revs for the Analytics refinery on stat1002 to free space (git fat jars replicating after each deployment, known issue) |
[production] |
| 02:46 |
<bd808> |
Loadavg on mw1198 very high (44+) and nginx/hhvm checks flapping |
[production] |
|
2017-06-01
§
|
| 23:33 |
<twentyafterfour> |
phabricator upgrade complete. |
[production] |
| 23:29 |
<twentyafterfour> |
Performing phabricator update, expect momentary downtime. |
[production] |
| 23:25 |
<twentyafterfour> |
Preparing phabricator update to tag release/2017-06-01/1 [ https://phabricator.wikimedia.org/project/view/2802/ ] |
[production] |
| 23:20 |
<ebernhardson@tin> |
Synchronized wmf-config/CirrusSearch-common.php: T163463: apply sister search restrictions requested by enwiki (duration: 00m 39s) |
[production] |
| 23:18 |
<ebernhardson@tin> |
Synchronized wmf-config/InitialiseSettings.php: T163463: apply sister search restrictions requested by enwiki (duration: 00m 40s) |
[production] |
| 21:59 |
<RainbowSprinkles> |
gerrit2001: Upgraded to 2.13.8, seems to be running fine this time. |
[production] |
| 20:37 |
<mobrovac@tin> |
Finished deploy [citoid/deploy@ba0db9c]: Update spec to minimise alert noise - T163986 (duration: 05m 20s) |
[production] |
| 20:32 |
<mobrovac@tin> |
Started deploy [citoid/deploy@ba0db9c]: Update spec to minimise alert noise - T163986 |
[production] |
| 20:23 |
<bsitzmann@tin> |
Finished deploy [mobileapps/deploy@2a8e648]: Update mobileapps to c4dc72d (duration: 05m 18s) |
[production] |
| 20:18 |
<bsitzmann@tin> |
Started deploy [mobileapps/deploy@2a8e648]: Update mobileapps to c4dc72d |
[production] |
| 19:30 |
<mepps> |
updated SmashPig from 4f84d88d3acaf8e54c5a18a7c76084ba78eb145b to d4458fa552d066f52a9324c09ff8871fcdb89864 |
[production] |
| 19:25 |
<thcipriani@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.30.0-wmf.2 |
[production] |
| 19:23 |
<gehel@tin> |
Finished deploy [wdqs/wdqs@3936e36]: (no justification provided) (duration: 01m 20s) |
[production] |
| 19:22 |
<gehel@tin> |
Started deploy [wdqs/wdqs@3936e36]: (no justification provided) |
[production] |
| 19:08 |
<thcipriani@tin> |
Synchronized wmf-config/CommonSettings.php: [[gerrit:356589|Revert "Add RejectParserCacheValue handler for mw-parser-output invalidation"]] T166345 (duration: 00m 43s) |
[production] |
| 18:20 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=wdqs1002.eqiad.wmnet |
[production] |
| 18:20 |
<gehel> |
wdqs1002 back in LVS - thermal paste added - T166524 |
[production] |
| 17:41 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=wdqs1002.eqiad.wmnet |
[production] |
| 17:41 |
<gehel> |
shutting down wdqs1002 for maintenance - T166524 |
[production] |
| 17:02 |
<elukey> |
sto mysql, eventlogging_sync and shutdown db1047 (analytics-store) for maintenance - T159266 |
[production] |
| 16:22 |
<jynus> |
retrying reimage of db2044 |
[production] |
| 15:03 |
<elukey> |
restart kafka100[23] for jvm upgrades |
[production] |
| 14:21 |
<mforns@tin> |
Finished deploy [analytics/refinery@7540403]: (no justification provided) (duration: 02m 50s) |
[production] |
| 14:18 |
<mforns@tin> |
Started deploy [analytics/refinery@7540403]: (no justification provided) |
[production] |
| 14:00 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2048 after maintenance (duration: 00m 44s) |
[production] |
| 13:18 |
<marostegui> |
Deploy alter table s3 revision on labsdb1001 - T166278 |
[production] |
| 13:15 |
<marostegui> |
Deploy alter table s3 revision on labsdb1011 - T166278 |
[production] |
| 13:11 |
<gilles> |
restored original configuration on mwdebug1001 |
[production] |
| 11:33 |
<godog> |
test upgrade of swift 2.10 on ms-fe2005 - T162609 |
[production] |
| 10:24 |
<gilles> |
Point nutcracker to localhost on mwdebug1001 |
[production] |
| 10:06 |
<godog> |
run puppet to blacklist acpi_power_meter across the fleet and rmmod the module |
[production] |
| 09:51 |
<_joe_> |
refreshing facts on the puppet compiler |
[production] |
| 08:15 |
<godog> |
upgrade grafana to 4.3.2 on labmon1001 / krypton |
[production] |
| 07:49 |
<gilles> |
editing wikiversions.php manually on mwdebug1001 to point enwiki to wmf.2 |
[production] |
| 06:08 |
<marostegui> |
Deploy alter table on s3, labsdb1010 - T166278 |
[production] |
| 06:07 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1035 - T166278 (duration: 00m 57s) |
[production] |
| 06:04 |
<marostegui> |
Deploy alter table on s3, db1044 - T166278 |
[production] |
| 06:02 |
<marostegui> |
Deploy alter table on s3, dbstore1001 - T166278 |
[production] |
| 05:58 |
<elukey> |
powercycle cp3032 - T166758 |
[production] |