|
2020-04-27
ยง
|
| 13:41 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 13:38 |
<_joe_> |
repooling both mw1407 and mw1409 for tesing T99740 |
[production] |
| 13:30 |
<_joe_> |
depooled mw1409 as well as mw1407 for further benchmarking, T99740 |
[production] |
| 13:28 |
<otto@deploy1001> |
helmfile [CODFW] Ran 'apply' command on namespace 'eventgate-analytics' for release 'canary' . |
[production] |
| 13:28 |
<otto@deploy1001> |
helmfile [CODFW] Ran 'apply' command on namespace 'eventgate-analytics' for release 'production' . |
[production] |
| 13:10 |
<elukey> |
roll restart elastic on cloudelastic-chi again to pick up new JVM settings - T231517 |
[production] |
| 13:09 |
<marostegui> |
Deploy schema change on s7 codfw, lag will show up - T250055 |
[production] |
| 12:53 |
<marostegui> |
Drop T248086_wb_terms from db1104 - T248086 |
[production] |
| 12:50 |
<marostegui> |
Removed img_deleted from s1 (enwiki) T250055 |
[production] |
| 12:49 |
<akosiaris> |
rolling back etherpad to 1.8.0 |
[production] |
| 12:45 |
<akosiaris> |
upgrade etherpad to 1.8.3 |
[production] |
| 12:41 |
<marostegui> |
Remove empty table T248086_wb_terms from wikidatawiki on s3 eqiad - T248086 |
[production] |
| 12:36 |
<marostegui> |
Remove empty table T248086_wb_terms from wikidatawiki on s3 codfw master - T248086 |
[production] |
| 12:32 |
<marostegui> |
Remove empty table T248086_wb_terms from wikidatawiki on s8 codfw master - T248086 |
[production] |
| 12:14 |
<marostegui> |
Remove empty table T248086_wb_terms from commonswiki and testcommonswiki on s4 master - T248086 |
[production] |
| 12:06 |
<ema> |
cp: upgrade purged to 0.8 T249583 |
[production] |
| 11:53 |
<Lucas_WMDE> |
EU SWAT done |
[production] |
| 11:49 |
<hoo> |
Started Wikibase rebuildItemsPerSite on mwmaint1002 for wikidatawiki. Can be killed at any time, if necessary. (T249613) |
[production] |
| 11:46 |
<lucaswerkmeister-wmde@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:592515|Enable cross-project search on frwiktionary (T250724)]] (duration: 00m 57s) |
[production] |
| 11:41 |
<lucaswerkmeister-wmde@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:592330|Add transwiki import sources in zhwiki (T250972)]] (duration: 00m 57s) |
[production] |
| 11:25 |
<addshore> |
repool wdqs1007 T242453 |
[production] |
| 11:21 |
<lucaswerkmeister-wmde@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:592306|Add two domains in wgCopyUploadsDomains (T250903, T250904)]] (duration: 00m 57s) |
[production] |
| 11:20 |
<_joe_> |
restarted php-fpm on mw1407 to pick up enlarged opcache values, T99740 |
[production] |
| 11:14 |
<jdrewniak@deploy1001> |
Synchronized portals: Wikimedia Portals Update: [[gerrit:592634| Bumping portals to master (563985)]] (duration: 00m 57s) |
[production] |
| 11:13 |
<jdrewniak@deploy1001> |
Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:592634| Bumping portals to master (563985)]] (duration: 00m 58s) |
[production] |
| 11:09 |
<cparle@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: [SDC] Enable constraints on production commons (duration: 00m 57s) |
[production] |
| 11:08 |
<cparle@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: [SDC] Enable constraints on production commons (duration: 00m 58s) |
[production] |
| 10:52 |
<hoo> |
Running the pruneItemsPerSite on mwmaint1002 maintenance script for Wikidata (T249613) |
[production] |
| 10:52 |
<hoo@deploy1001> |
Synchronized php-1.35.0-wmf.28/extensions/Wikibase: pruneItemsPerSite: Fix join_condition call signature (T249613) (duration: 01m 02s) |
[production] |
| 10:49 |
<hoo@deploy1001> |
Synchronized php-1.35.0-wmf.28/extensions/Wikibase: pruneItemsPerSite: Fix join_condition call signature (T249613) (duration: 01m 01s) |
[production] |
| 10:32 |
<mutante> |
contint2001 - systemd status was degraded. icinga alerted. failed unit was jenkins. starting it failed with "address already in use". manually started without using systemctl? killed jenkins and started again with systemctl. T224591 |
[production] |
| 10:29 |
<mutante> |
contint2001 - jenkins failed and can't start because address is already in use |
[production] |
| 10:22 |
<addshore> |
depool and restart wdqs1007 (deadlocks) T242453 |
[production] |
| 09:54 |
<hoo@deploy1001> |
Synchronized php-1.35.0-wmf.28/extensions/Wikibase: Add pruneItemsPerSite maintenance script (T249613) (duration: 01m 06s) |
[production] |
| 09:34 |
<jynus@cumin2001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 09:34 |
<jynus@cumin2001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 09:34 |
<jynus@cumin2001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 09:33 |
<jynus@cumin2001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 09:33 |
<jynus@cumin2001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 09:32 |
<jynus@cumin2001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 09:32 |
<jynus@cumin2001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 09:31 |
<jynus@cumin2001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 09:25 |
<marostegui> |
Stop MySQL on labsdb1012 to reclone labsdb1011 - T249188 |
[production] |
| 09:11 |
<marostegui> |
Deploy schema change on s1 codfw, lag will show up - T250055 |
[production] |
| 08:52 |
<moritzm> |
restarting cas on idp1001 to pick up Java 11 security update (will void active SSO sessions) |
[production] |
| 08:26 |
<marostegui> |
Deploy schema change on s5 codfw, lag will show up - T250055 |
[production] |
| 08:24 |
<kormat> |
Truncating and optimizing parsercache for pc1010 and pc2010 T247787 |
[production] |
| 08:18 |
<mutante> |
running puppet on all cp-ats |
[production] |
| 08:15 |
<godog> |
add 80G to prometheus global LV |
[production] |
| 07:25 |
<elukey> |
roll restart elastic-chi on cloudelastic100[1-4] to pick up the last JVM GC settings - T231517 |
[production] |