|
2017-06-07
§
|
| 08:28 |
<gehel> |
upgrading kibana to v5.4.1 on deployment-logstash2 |
[releng] |
| 08:02 |
<marostegui> |
Run redact_sanitarium on db1095 for dewiki - T153743 |
[production] |
| 07:22 |
<marostegui> |
Deploy alter table on db1047 enwiki.revision - T162807 |
[production] |
| 06:50 |
<elukey> |
restarted mediacounts-archive-wf-2017-06-06 in Hue (Java OOMs) |
[analytics] |
| 06:49 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1056 - T166206 (duration: 00m 44s) |
[production] |
| 05:35 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1053, depool db1056 - T166206 (duration: 01m 03s) |
[production] |
| 03:11 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Wed Jun 7 03:11:40 UTC 2017 (duration 6m 54s) |
[production] |
| 03:04 |
<l10nupdate@tin> |
scap sync-l10n completed (1.30.0-wmf.4) (duration: 14m 29s) |
[production] |
| 02:30 |
<l10nupdate@tin> |
scap sync-l10n completed (1.30.0-wmf.2) (duration: 07m 57s) |
[production] |
| 00:21 |
<RainbowSprinkles> |
gerrit: rolled back to 2.13.4-13-gc0c5cc4742 from 2.13.8. T152640 rearing its ugly head again (login issues) |
[production] |
|
2017-06-06
§
|
| 23:59 |
<thcipriani@tin> |
Synchronized php-1.30.0-wmf.2/extensions/Flow/includes/Content/BoardContentHandler.php: SWAT: [[gerrit:357500|Revert "Throw when unserializing invalid Flow workflow metadata JSON"]] T166100 T156813 (duration: 00m 43s) |
[production] |
| 23:58 |
<thcipriani@tin> |
Synchronized php-1.30.0-wmf.4/extensions/Flow/includes/Content/BoardContentHandler.php: SWAT: [[gerrit:357501|Revert "Throw when unserializing invalid Flow workflow metadata JSON"]] T166100 T156813 (duration: 00m 45s) |
[production] |
| 23:56 |
<RainbowSprinkles> |
gerrit: back from reindexing |
[production] |
| 23:55 |
<RainbowSprinkles> |
gerrit: force stopping for a second to reindex accounts |
[production] |
| 23:17 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:357425|Disable page previews on wikispecies]] T166894 (duration: 00m 44s) |
[production] |
| 23:12 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:357426|Update ContentNamespaces for Commons Wiki]] T167077 (duration: 00m 46s) |
[production] |
| 22:23 |
<bd808> |
Fixed webservice startup crash by removing error.log file with bad ownership (-rw-r--r-- ahecht:tools.excel2wiki) |
[tools.excel2wiki] |
| 21:57 |
<RainbowSprinkles> |
gerrit: restarting last time, didn't work like I wanted |
[production] |
| 21:53 |
<RainbowSprinkles> |
gerrit: restarting to test a config tweak |
[production] |
| 21:41 |
<mutante> |
contint1001 - graceful'ed Apache to deploy gerrit:351391 |
[production] |
| 21:19 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: unbreak mw.org pref page |
[production] |
| 20:30 |
<chasemp> |
rebooting tools-bastion-02 as unresponsive (up 76 days and lots of seemingly left behind things running) |
[tools] |
| 20:21 |
<RainbowSprinkles> |
gerrit: Down for just a moment, finally doing point release on cobalt |
[production] |
| 19:57 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group0 to wmf.4 |
[production] |
| 19:45 |
<demon@tin> |
Finished scap: testwiki to wmf.4 + prepping l10n. again (x2) (duration: 20m 25s) |
[production] |
| 19:36 |
<mutante> |
cobalt - removed systemd unit file (that has issues with ulimit and isn't used yet) - ran "systemctl reset-failed" which cleared the "systemctl status" which made the Icinga check recover |
[production] |
| 19:24 |
<demon@tin> |
Started scap: testwiki to wmf.4 + prepping l10n. again (x2) |
[production] |
| 19:23 |
<demon@tin> |
scap failed: RuntimeError scap failed: average error rate on 1/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/3888cca979647b9381a7739b0bdbc88e for details) (duration: 13m 32s) |
[production] |
| 19:23 |
<demon@tin> |
scap failed: average error rate on 1/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/3888cca979647b9381a7739b0bdbc88e for details) |
[production] |
| 19:10 |
<demon@tin> |
Started scap: testwiki to wmf.4 + prepping l10n. again |
[production] |
| 19:08 |
<demon@tin> |
Synchronized README: No-op, just forcing co-master sync (duration: 01m 27s) |
[production] |
| 19:01 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: testwiki back to wmf.2 |
[production] |
| 18:55 |
<maxsem@tin> |
Finished scap: LoginNotify to testwiki - rebuild messages (duration: 38m 19s) |
[production] |
| 18:16 |
<maxsem@tin> |
Started scap: LoginNotify to testwiki - rebuild messages |
[production] |
| 18:15 |
<maxsem@tin> |
Synchronized wmf-config: https://gerrit.wikimedia.org/r/#/c/357317/2 (duration: 00m 44s) |
[production] |
| 18:10 |
<maxsem@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/357317/2 (duration: 00m 44s) |
[production] |
| 18:03 |
<demon@tin> |
Finished scap: testwiki to wmf.3, prepping l10n cache (duration: 31m 58s) |
[production] |
| 17:31 |
<demon@tin> |
Started scap: testwiki to wmf.3, prepping l10n cache |
[production] |
| 16:53 |
<moritzm> |
installing wireshark security updates on trusty (jessie already fixed) |
[production] |
| 16:41 |
<bblack> |
rebooted lvs1007 (kernel update) |
[production] |
| 16:35 |
<bblack> |
rebooted lvs1007 (kernel update) |
[production] |
| 15:44 |
<ottomata> |
restarting eventlogging mysql consumer to allow is_mediawiki events through is_not_bot filter |
[analytics] |
| 15:24 |
<ottomata> |
restarting eventlogging processor to bring in is_mediawiki ua classification |
[analytics] |
| 15:21 |
<otto@tin> |
Finished deploy [eventlogging/analytics@37233cd]: (no justification provided) (duration: 00m 04s) |
[production] |
| 15:21 |
<otto@tin> |
Started deploy [eventlogging/analytics@37233cd]: (no justification provided) |
[production] |
| 14:58 |
<moritzm> |
installing libsndfile security updates on trusty |
[production] |
| 14:34 |
<hashar> |
deleting buildlog.integration.eqiad.wmflabs was mean to receive Jenkins logs in ElasticSearch. We are experimenting with relforge1001.eqiad.wmnet now - T78705 |
[releng] |
| 14:01 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Restore db1094 original weight (duration: 00m 40s) |
[production] |
| 13:46 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase db1094 weight (duration: 00m 40s) |
[production] |
| 13:39 |
<elukey> |
shutdown analytics1033 and analytics1039 to replace their BBU - T166140 |
[production] |