|
2017-07-26
ยง
|
| 19:48 |
<demon@tin> |
Started scap: group1 to wmf.11 |
[production] |
| 19:47 |
<demon@tin> |
scap failed: RuntimeError scap failed: average error rate on 1/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/2cc7028226a539553178454fc2f14459 for details) (duration: 12m 03s) |
[production] |
| 19:47 |
<demon@tin> |
scap failed: average error rate on 1/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/2cc7028226a539553178454fc2f14459 for details) |
[production] |
| 19:35 |
<demon@tin> |
Started scap: group1 to wmf.11 |
[production] |
| 19:30 |
<mutante> |
mx1001 - temp disable puppet to test adjusted sudo privileges for an icinga check |
[production] |
| 19:24 |
<ejegg> |
disabled queue consumers for CiviCRM update |
[production] |
| 19:06 |
<bblack> |
cp1074: run-no-puppet varnish-backend-restart (mailbox lag in icinga) |
[production] |
| 19:01 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=wdqs1001.eqiad.wmnet |
[production] |
| 19:01 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=wdqs1001.wmnet |
[production] |
| 19:01 |
<gehel> |
depooling wdqs1001 for data reload - T166244 |
[production] |
| 18:50 |
<niharika29@tin> |
Synchronized php-1.30.0-wmf.11/resources/src/mediawiki.rcfilters/: RCFilters: Followup I78e23f85c3: Don't disable RCFilters system when fetching results https://gerrit.wikimedia.org/r/#/c/367850/ (duration: 00m 46s) |
[production] |
| 18:35 |
<niharika29@tin> |
Synchronized php-1.30.0-wmf.11/resources/src/mediawiki.rcfilters/: RCFilters: Improve loading animation https://gerrit.wikimedia.org/r/#/c/367833/, RCFilters UI: Unbreak limit and days widgets in non-experimental mode https://gerrit.wikimedia.org/r/#/c/367837/ (duration: 00m 45s) |
[production] |
| 18:20 |
<niharika29@tin> |
Synchronized wmf-config/InitialiseSettings.php: Create 'rollbacker' user group in frwiki https://gerrit.wikimedia.org/r/#/c/365538/ (duration: 00m 47s) |
[production] |
| 17:46 |
<bblack> |
nitrogen: disabled puppet agent, manually hacked puppetdb.service unit file, restarted puppetdb.service... |
[production] |
| 17:12 |
<moritzm> |
restarting gerrit to pick up Java security update |
[production] |
| 17:11 |
<moritzm> |
installing openjdk-8 security updates on cobalt and removing unused openjdk-7 packages |
[production] |
| 16:55 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2070 (duration: 00m 45s) |
[production] |
| 16:27 |
<jynus> |
upgrading and rebooting db2070 |
[production] |
| 16:22 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2069, depool db2070 (duration: 00m 45s) |
[production] |
| 16:14 |
<moritzm> |
upgraded nodejs on restbase* |
[production] |
| 15:48 |
<jynus> |
upgrade and reboot db2069 |
[production] |
| 15:45 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2068, depool db2069, pool db2072 with more weight (duration: 00m 46s) |
[production] |
| 15:39 |
<moritzm> |
rolling upgrade/service restarts of nodejs in eqiad |
[production] |
| 15:32 |
<andrewbogott> |
patching puppetmaster1001, possible puppet hiccups coming up |
[production] |
| 15:29 |
<moritzm> |
upgrade nodejs on remaining scb hosts (along with service restarts) |
[production] |
| 15:20 |
<moritzm> |
upgrade nodejs on scb2001 (currently depooled for testing) |
[production] |
| 15:17 |
<jynus> |
restarting and upgrading db2068 |
[production] |
| 15:15 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Depool db2068 (duration: 00m 46s) |
[production] |
| 14:40 |
<moritzm> |
installing spice security updates |
[production] |
| 14:21 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Pool db2072 (duration: 00m 45s) |
[production] |
| 13:44 |
<gehel> |
restarting cassandra on maps clusters |
[production] |
| 13:36 |
<zeljkof> |
EU SWAT finished |
[production] |
| 13:34 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:367889|Revert eswikisource paths due to oversized logos (T170604)]] (duration: 00m 46s) |
[production] |
| 13:24 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:367676|HD logos for eswikivoyage and added some missing paths to the config (T170604)]] (duration: 00m 46s) |
[production] |
| 13:22 |
<zfilipin@tin> |
Synchronized static/images/project-logos/: SWAT: [[gerrit:367676|HD logos for eswikivoyage and added some missing paths to the config (T170604)]] (duration: 00m 54s) |
[production] |
| 12:12 |
<moritzm> |
installing xorg-server updates from jessie 8.9 point release |
[production] |
| 11:02 |
<hoo> |
Updated the Wikidata property suggester with data from Monday's JSON dump and applied the T132839 workarounds |
[production] |
| 09:58 |
<jmm@puppetmaster1001> |
conftool action : set/pooled=no; selector: mw2119.codfw.wmnet |
[production] |
| 09:58 |
<jmm@puppetmaster1001> |
conftool action : set/pooled=no; selector: mw2119.codfw.wmnet |
[production] |
| 09:58 |
<jmm@puppetmaster1001> |
conftool action : set/pooled=yes; selector: mw2119.codfw.wmnet |
[production] |
| 09:57 |
<moritzm> |
reimaging mw2152 to jessie (T145742) |
[production] |
| 09:52 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1066 after maintenance (duration: 00m 46s) |
[production] |
| 08:47 |
<elukey> |
rollout logster 0.0.10-2~jessie1 to the cache hosts |
[production] |
| 08:46 |
<elukey> |
upload logster 0.0.10-2~jessie1 to jessie-wikimedia |
[production] |
| 08:42 |
<jynus> |
upgrading and restarting db1066 |
[production] |
| 08:26 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1066 for maintenance (duration: 00m 46s) |
[production] |
| 08:26 |
<moritzm> |
reimaging mw2119 to jessie (T145742) |
[production] |
| 08:02 |
<moritzm> |
installing Java security updates on jessie-based stat systems |
[production] |
| 07:59 |
<moritzm> |
restarting cassandra-metrics-collector on maps* to pick up openjdk security update |
[production] |
| 07:56 |
<moritzm> |
restarting cassandra-metrics-collector on restbase* to pick up openjdk security update |
[production] |