|
2019-04-12
§
|
| 23:34 |
<mutante> |
- toolsbeta-k8s-master-01 - was out of disk space on / , puppet failed to run because out of disk, rename existing syslog.1.gz, gzip syslog.1, rename existing daemon.log.1.gz, gzip daemong.log.1 |
[toolsbeta] |
| 21:16 |
<Krinkle> |
scap was unable to sync to 1 apache (connect to host cloudweb2001-dev.wikimedia.org port 22: Connection timed out) |
[production] |
| 21:10 |
<krinkle@deploy1001> |
Synchronized php-1.33.0-wmf.25/extensions/ImageMap/includes/ImageMap.php: I0ee84f059da / T217087 (duration: 05m 12s) |
[production] |
| 19:27 |
<dzahn@cumin1001> |
END (FAIL) - Cookbook sre.hosts.decommission (exit_code=99) |
[production] |
| 19:27 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 19:24 |
<dzahn@cumin1001> |
END (FAIL) - Cookbook sre.hosts.decommission (exit_code=99) |
[production] |
| 19:24 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 18:59 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 18:59 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 17:17 |
<onimisionipe> |
depooling maps2002 for postgres init |
[production] |
| 17:16 |
<onimisionipe> |
repooling maps2001 - postgres init is complete |
[production] |
| 17:09 |
<Lucas_WMDE> |
wikidata-shex for i in {1..36}; do printf 'Schema:O%d\n' "$i"; done | mwscript purgePage.php --skip-exists-check # purge Schema pages after config change |
[wikidata-dev] |
| 17:08 |
<Lucas_WMDE> |
wikidata-shex add packages/shex-webapp/ in wgWBSchemaShExSimpleUrl |
[wikidata-dev] |
| 17:06 |
<Lucas_WMDE> |
wikidata-shex for i in {1..36}; do printf 'Schema:O%d\n' "$i"; done | mwscript purgePage.php --skip-exists-check # purge Schema pages after config change (T218886) |
[wikidata-dev] |
| 17:05 |
<Lucas_WMDE> |
wikidata-shex add hideData and textMapIsSparqlQuery to wgWBSchemaShExSimpleUrl (T218886) |
[wikidata-dev] |
| 16:14 |
<elukey> |
install ifstat on all the mc1* hosts for network bandwidth investigation |
[production] |
| 15:56 |
<gehel> |
starting data trasnfer from wdqs1008 to wdqs1009 - T220830 |
[production] |
| 15:32 |
<thcipriani> |
gerrit back |
[production] |
| 15:29 |
<thcipriani> |
gerrit restart incoming |
[production] |
| 15:09 |
<Krenair> |
upload traffic now through cache-upload05 |
[releng] |
| 14:29 |
<onimisionipe> |
depool maps2001 for postgres initialization |
[production] |
| 13:24 |
<akosiaris> |
re-enable puppet across the fleet. Patch merged, recovery storm coming |
[production] |
| 13:18 |
<akosiaris> |
disable puppet across the fleet to avoid incoming puppet alert storm |
[production] |
| 12:57 |
<marostegui> |
Purge old rows and optimize tables on spare host pc1010 T210725 |
[production] |
| 12:53 |
<urandom> |
decommissioning cassandra-c, restbase2008 -- T208087 |
[production] |
| 12:49 |
<gehel> |
rolling restart of cassandra on maps* for jvm upgrade |
[production] |
| 12:22 |
<arturo> |
T220095 disable icinga checks for labtestcontrol2003 |
[production] |
| 12:16 |
<gilles@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T220807 Reduce cawiki survey sampling rate (duration: 05m 11s) |
[production] |
| 11:56 |
<moritzm> |
upgrading app server canaries to version 1.8.1 of the PHP wikidiff extension (HHVM already deployed) T203069 |
[production] |
| 11:46 |
<moritzm> |
upgrading acmechief hosts to latest buster state |
[production] |
| 11:44 |
<gilles@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T220807 Oversample navtiming on cawiki and commonswiki (duration: 05m 14s) |
[production] |
| 11:37 |
<Trey314159> |
reindexing Greek, Turkish, and Irish wikis on elastic@eqiad and elastic@codfw complete (T217806) |
[production] |
| 11:19 |
<moritzm> |
installed Java security updates on relforge* hosts |
[production] |
| 11:10 |
<moritzm> |
installing Java security updates on remaining maps hosts |
[production] |
| 10:32 |
<arturo> |
T219626 reimaging cloudcontrol2001-dev |
[production] |
| 10:13 |
<elukey> |
matomo updated to 3.9.1 on matomo1001 + deb upload to wikimedia-stretch - T218037 |
[production] |
| 10:12 |
<elukey> |
matomo upgraded to 3.9.1 to fix some security vulns |
[analytics] |
| 10:04 |
<JeanFred> |
Started a new harvest to better investigate T180833 |
[tools.heritage] |
| 09:53 |
<moritzm> |
updated mwdebug1001 to php-wikidiff 1.8.1 |
[production] |
| 09:37 |
<moritzm> |
updated mwdebug1002 to php-wikidiff 1.8.1 |
[production] |
| 09:30 |
<volans> |
reset mgmt card on labtestcontrol2003 - T220783 |
[production] |
| 09:07 |
<moritzm> |
added the wikimedia repository key to the stretch build chroot on boron, fixes builds using the PHP72/SPICERACK hooks |
[production] |
| 09:05 |
<arturo> |
T218021 disable icinga checks for labtestcontrol2001 |
[production] |
| 08:35 |
<gilles@deploy1001> |
Synchronized php-1.33.0-wmf.25/extensions/NavigationTiming/modules/ext.navigationTiming.js: T220788 Fix veaction === null case (duration: 00m 54s) |
[production] |
| 08:02 |
<moritzm> |
updated ssacli in thirdparty/hwraid component for stretch to 3.30-13.0 T220787 |
[production] |
| 07:12 |
<marostegui> |
Manually install ssacli on db2[097|098|099|100|101|102] T220787 T220572 |
[production] |