|
2017-06-07
§
|
| 09:28 |
<gehel> |
upgrading kibana to v5.3.3 on logstash cluster - T167266 |
[production] |
| 09:15 |
<ema> |
upgrade lvs4001-4004 to jessie 8.8 point release T164703 |
[production] |
| 08:58 |
<marostegui> |
Deploy alter table on s2 - db1076 - T166205 |
[production] |
| 08:58 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1076 - T166205 (duration: 00m 43s) |
[production] |
| 08:50 |
<marostegui> |
Deploy alter table s4 - db1056 - T166206 |
[production] |
| 08:34 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1056 - T166206 (duration: 00m 43s) |
[production] |
| 08:02 |
<marostegui> |
Run redact_sanitarium on db1095 for dewiki - T153743 |
[production] |
| 07:22 |
<marostegui> |
Deploy alter table on db1047 enwiki.revision - T162807 |
[production] |
| 06:49 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1056 - T166206 (duration: 00m 44s) |
[production] |
| 05:35 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1053, depool db1056 - T166206 (duration: 01m 03s) |
[production] |
| 03:11 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Wed Jun 7 03:11:40 UTC 2017 (duration 6m 54s) |
[production] |
| 03:04 |
<l10nupdate@tin> |
scap sync-l10n completed (1.30.0-wmf.4) (duration: 14m 29s) |
[production] |
| 02:30 |
<l10nupdate@tin> |
scap sync-l10n completed (1.30.0-wmf.2) (duration: 07m 57s) |
[production] |
| 00:21 |
<RainbowSprinkles> |
gerrit: rolled back to 2.13.4-13-gc0c5cc4742 from 2.13.8. T152640 rearing its ugly head again (login issues) |
[production] |
|
2017-06-06
§
|
| 23:59 |
<thcipriani@tin> |
Synchronized php-1.30.0-wmf.2/extensions/Flow/includes/Content/BoardContentHandler.php: SWAT: [[gerrit:357500|Revert "Throw when unserializing invalid Flow workflow metadata JSON"]] T166100 T156813 (duration: 00m 43s) |
[production] |
| 23:58 |
<thcipriani@tin> |
Synchronized php-1.30.0-wmf.4/extensions/Flow/includes/Content/BoardContentHandler.php: SWAT: [[gerrit:357501|Revert "Throw when unserializing invalid Flow workflow metadata JSON"]] T166100 T156813 (duration: 00m 45s) |
[production] |
| 23:56 |
<RainbowSprinkles> |
gerrit: back from reindexing |
[production] |
| 23:55 |
<RainbowSprinkles> |
gerrit: force stopping for a second to reindex accounts |
[production] |
| 23:17 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:357425|Disable page previews on wikispecies]] T166894 (duration: 00m 44s) |
[production] |
| 23:12 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:357426|Update ContentNamespaces for Commons Wiki]] T167077 (duration: 00m 46s) |
[production] |
| 21:57 |
<RainbowSprinkles> |
gerrit: restarting last time, didn't work like I wanted |
[production] |
| 21:53 |
<RainbowSprinkles> |
gerrit: restarting to test a config tweak |
[production] |
| 21:41 |
<mutante> |
contint1001 - graceful'ed Apache to deploy gerrit:351391 |
[production] |
| 21:19 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: unbreak mw.org pref page |
[production] |
| 20:21 |
<RainbowSprinkles> |
gerrit: Down for just a moment, finally doing point release on cobalt |
[production] |
| 19:57 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group0 to wmf.4 |
[production] |
| 19:45 |
<demon@tin> |
Finished scap: testwiki to wmf.4 + prepping l10n. again (x2) (duration: 20m 25s) |
[production] |
| 19:36 |
<mutante> |
cobalt - removed systemd unit file (that has issues with ulimit and isn't used yet) - ran "systemctl reset-failed" which cleared the "systemctl status" which made the Icinga check recover |
[production] |
| 19:24 |
<demon@tin> |
Started scap: testwiki to wmf.4 + prepping l10n. again (x2) |
[production] |
| 19:23 |
<demon@tin> |
scap failed: RuntimeError scap failed: average error rate on 1/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/3888cca979647b9381a7739b0bdbc88e for details) (duration: 13m 32s) |
[production] |
| 19:23 |
<demon@tin> |
scap failed: average error rate on 1/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/3888cca979647b9381a7739b0bdbc88e for details) |
[production] |
| 19:10 |
<demon@tin> |
Started scap: testwiki to wmf.4 + prepping l10n. again |
[production] |
| 19:08 |
<demon@tin> |
Synchronized README: No-op, just forcing co-master sync (duration: 01m 27s) |
[production] |
| 19:01 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: testwiki back to wmf.2 |
[production] |
| 18:55 |
<maxsem@tin> |
Finished scap: LoginNotify to testwiki - rebuild messages (duration: 38m 19s) |
[production] |
| 18:16 |
<maxsem@tin> |
Started scap: LoginNotify to testwiki - rebuild messages |
[production] |
| 18:15 |
<maxsem@tin> |
Synchronized wmf-config: https://gerrit.wikimedia.org/r/#/c/357317/2 (duration: 00m 44s) |
[production] |
| 18:10 |
<maxsem@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/357317/2 (duration: 00m 44s) |
[production] |
| 18:03 |
<demon@tin> |
Finished scap: testwiki to wmf.3, prepping l10n cache (duration: 31m 58s) |
[production] |
| 17:31 |
<demon@tin> |
Started scap: testwiki to wmf.3, prepping l10n cache |
[production] |
| 16:53 |
<moritzm> |
installing wireshark security updates on trusty (jessie already fixed) |
[production] |
| 16:41 |
<bblack> |
rebooted lvs1007 (kernel update) |
[production] |
| 16:35 |
<bblack> |
rebooted lvs1007 (kernel update) |
[production] |
| 15:21 |
<otto@tin> |
Finished deploy [eventlogging/analytics@37233cd]: (no justification provided) (duration: 00m 04s) |
[production] |
| 15:21 |
<otto@tin> |
Started deploy [eventlogging/analytics@37233cd]: (no justification provided) |
[production] |
| 14:58 |
<moritzm> |
installing libsndfile security updates on trusty |
[production] |
| 14:01 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Restore db1094 original weight (duration: 00m 40s) |
[production] |
| 13:46 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase db1094 weight (duration: 00m 40s) |
[production] |
| 13:39 |
<elukey> |
shutdown analytics1033 and analytics1039 to replace their BBU - T166140 |
[production] |
| 13:33 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1094 with low weight (duration: 00m 40s) |
[production] |