|
2018-10-18
§
|
| 22:00 |
<mutante> |
lvs1011,lvs1012 - manually editing nagios NRPE config and restarting service (to make monitoring from icinga1001 work and puppet is disabled) |
[production] |
| 21:52 |
<mutante> |
eeden - manually editing nagios NRPE config and restarting service (to make monitoring from icinga1001 work and puppet is disabled) |
[production] |
| 21:49 |
<twentyafterfour@deploy1001> |
rebuilt and synchronized wikiversions files: group2 wikis to 1.32.0-wmf.26 refs T191072 |
[production] |
| 21:46 |
<twentyafterfour@deploy1001> |
Synchronized php-1.32.0-wmf.26/includes/filerepo/file/LocalFile.php: sync Id97e1c7c2655d90928c777bc3377e5ea23f49f6b refs T207419 (duration: 00m 53s) |
[production] |
| 21:29 |
<twentyafterfour@deploy1001> |
Synchronized php-1.32.0-wmf.26/includes/filerepo/file/LocalFile.php: sync https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/468470/ refs T207419 (duration: 00m 54s) |
[production] |
| 20:49 |
<twentyafterfour@deploy1001> |
rebuilt and synchronized wikiversions files: group2 wikis to 1.32.0-wmf.24 refs T191072 |
[production] |
| 20:39 |
<twentyafterfour@deploy1001> |
rebuilt and synchronized wikiversions files: all wikis to 1.32.0-wmf.26 |
[production] |
| 20:21 |
<volans> |
start ferm on db2042, it failed to start at reboot due to DNS resolution timeout |
[production] |
| 19:22 |
<ejegg> |
updated SmashPig standalone deploy from 5f21d3f2db to 581c685326 |
[production] |
| 19:21 |
<ejegg> |
updated payments-wiki from a3892e4ed3 to 06848600ed |
[production] |
| 19:17 |
<shdubsh> |
rebooting graphite1004 |
[production] |
| 19:11 |
<shdubsh> |
upping ring buffer size on graphite1004 in an attempt to mitigate dropped packets at the interface -- T196484 |
[production] |
| 19:02 |
<sbisson@deploy1001> |
Synchronized php-1.32.0-wmf.26/extensions/PageTriage/: SWAT: [[gerrit:468384|Use Main Object Stash for keeping track of PageTriage last use]] (duration: 00m 54s) |
[production] |
| 18:19 |
<awight> |
Restarting ORES services for T88997 |
[production] |
| 17:33 |
<ladsgroup@deploy1001> |
Finished deploy [ores/deploy@4ac4c8b]: Logstash support for ores: T181546 T169586 T168921 T181630 T205256 (duration: 23m 48s) |
[production] |
| 17:19 |
<herron> |
aborted enabling kafka on logstash elasticsearch cluster due to puppet errors. reverted change T206454 |
[production] |
| 17:09 |
<ladsgroup@deploy1001> |
Started deploy [ores/deploy@4ac4c8b]: Logstash support for ores: T181546 T169586 T168921 T181630 T205256 |
[production] |
| 17:00 |
<twentyafterfour@deploy1001> |
Synchronized php: group1 wikis to 1.32.0-wmf.26 refs T191072 (duration: 00m 53s) |
[production] |
| 16:59 |
<twentyafterfour@deploy1001> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.32.0-wmf.26 refs T191072 |
[production] |
| 16:57 |
<herron> |
enabling kafka on logstash elasticsearch cluster T206454 |
[production] |
| 16:55 |
<twentyafterfour@deploy1001> |
Synchronized php-1.32.0-wmf.26/extensions/WikibaseQualityConstraints/src/ServiceWiring.php: sync https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/WikibaseQualityConstraints/+/468352/ refs T207394 (duration: 00m 54s) |
[production] |
| 16:52 |
<mobrovac@deploy1001> |
Finished deploy [restbase/deploy@6c879fa]: Have 100% of traffic directed to Proton as well - T186748 (duration: 20m 52s) |
[production] |
| 16:31 |
<mobrovac@deploy1001> |
Started deploy [restbase/deploy@6c879fa]: Have 100% of traffic directed to Proton as well - T186748 |
[production] |
| 15:51 |
<XioNoX> |
trunk cloud-instances2-b-eqiad between asw-b-eqiad and asw2-b-eqiad |
[production] |
| 15:50 |
<cmjohnson1> |
disabling checks on cloudvirt1019 for maintenance |
[production] |
| 15:42 |
<twentyafterfour> |
twentyafterfour@deploy1001 Synchronized php: group1 wikis to 1.32.0-wmf.24 refs T191072 (duration: 00m 53s) |
[production] |
| 15:40 |
<twentyafterfour@deploy1001> |
Synchronized php: group1 wikis to 1.32.0-wmf.24 refs T191072 (duration: 00m 53s) |
[production] |
| 15:39 |
<twentyafterfour@deploy1001> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.32.0-wmf.24 refs T191072 |
[production] |
| 15:35 |
<twentyafterfour@deploy1001> |
scap failed: average error rate on 6/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/db09a36be5ed3e81155041f7d46ad040 for details) |
[production] |
| 14:46 |
<moritzm> |
installing tomcat8 security updates |
[production] |
| 14:34 |
<moritzm> |
remove labvirt1018 from debmonitor (T207317) |
[production] |
| 14:28 |
<godog> |
temporarily bump default socket receive memory to 1MB on graphite1001, restart statsd-proxy and statsite |
[production] |
| 14:22 |
<godog> |
begin reformat of ms-be2041 - T199198 |
[production] |
| 14:21 |
<banyek> |
shutting down mysql and powering down db2042 (T202051) |
[production] |
| 14:13 |
<godog> |
corrections to the statements above, graphite1004 not graphite1001 |
[production] |
| 14:11 |
<godog> |
ditto for statsite instances on graphite1001, temporarily bump receive socket memory to 1MB and bounce the service |
[production] |
| 14:08 |
<godog> |
temporarily bump receive socket memory for statsd-proxy on graphite1001 and bounce the service |
[production] |
| 13:51 |
<moritzm> |
installing libidn security updates |
[production] |
| 12:59 |
<moritzm> |
installing libssh security updates |
[production] |
| 12:55 |
<godog> |
bounce statsd-proxy on graphite1001 |
[production] |
| 11:59 |
<addshore> |
SWAT done |
[production] |
| 11:59 |
<addshore@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Wikidata.org: enable sense data type T203888 (duration: 00m 54s) |
[production] |
| 11:54 |
<mobrovac@deploy1001> |
Finished deploy [restbase/deploy@1041a02]: Disable onthisday check - T203588 (duration: 21m 23s) |
[production] |
| 11:54 |
<zfilipin@deploy1001> |
Synchronized tests/InitialiseSettingsTest.php: SWAT: [[gerrit:467412|Test if logo specified in wgLogo/wgLogoHD exists (T207053)]] (duration: 00m 53s) |
[production] |
| 11:49 |
<zfilipin@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:467453|Fix typo in IS.php: use ltwiki instead of ltwikipedia (T207081)]] (duration: 00m 54s) |
[production] |
| 11:39 |
<zfilipin@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:467509|Use testwikidatawiki instead of testwikidata in IS.php (T207089)]] (duration: 00m 53s) |
[production] |
| 11:33 |
<mobrovac@deploy1001> |
Started deploy [restbase/deploy@1041a02]: Disable onthisday check - T203588 |
[production] |
| 11:29 |
<zfilipin@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:468226|Use new wordmarks in uzwiki (T205226)]] (duration: 00m 53s) |
[production] |