|
2016-07-12
§
|
| 18:08 |
<demon@tin> |
Synchronized wmf-config/InitialiseSettings.php: turn cx back on (duration: 00m 29s) |
[production] |
| 17:44 |
<demon@tin> |
Synchronized php-1.28.0-wmf.8/extensions/ContentTranslation/: ping limiter fixes (duration: 00m 29s) |
[production] |
| 17:37 |
<jynus> |
out of band ALTER TABLE recentchanges ADD KEY `name_type_patrolled_timestamp` on db1054 T140108 |
[production] |
| 16:58 |
<demon@tin> |
Synchronized wmf-config/InitialiseSettings.php: prep pinglimiter config for content translation (duration: 00m 33s) |
[production] |
| 16:56 |
<Amir1> |
deploying ores f472f65 to scb |
[production] |
| 16:51 |
<Amir1> |
deploying ores f472f65 to scb2001 |
[production] |
| 16:42 |
<godog> |
disable puppet on ms-fe* and re-enable gradually to apply https://gerrit.wikimedia.org/r/#/c/298297/ |
[production] |
| 16:25 |
<mattflaschen@tin> |
Synchronized php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php: T119736: ForeignWikiRequest: Bail early for non-global users (duration: 00m 32s) |
[production] |
| 16:12 |
<demon@tin> |
Synchronized wmf-config/InitialiseSettings.php: Disable content translation, outage right now (duration: 00m 29s) |
[production] |
| 15:44 |
<bblack> |
cache nodes: salt manual removal of vm compaction cron via sed ( https://gerrit.wikimedia.org/r/298499 ) |
[production] |
| 15:33 |
<thcipriani@tin> |
Synchronized php-1.28.0-wmf.9/extensions/Echo/includes/ForeignWikiRequest.php: SWAT: [[gerrit:298429|ForeignWikiRequest: Bail early for non-global users (T119736)]] (duration: 00m 31s) |
[production] |
| 15:29 |
<thcipriani@tin> |
Synchronized portals: SWAT: [[gerrit:298476|Bumping portals to master (T128546)]] (duration: 00m 29s) |
[production] |
| 15:28 |
<thcipriani@tin> |
Synchronized portals/prod/wikipedia.org/assets: SWAT: [[gerrit:298476|Bumping portals to master (T128546)]] (duration: 00m 29s) |
[production] |
| 15:20 |
<bblack> |
upgrading nginx to 1.11.2-1+wmf1 on all caches |
[production] |
| 15:18 |
<thcipriani@tin> |
Synchronized static/images/project-logos/trwikimedia.png: SWAT: Revert [[gerrit:298441|Logo update for trwikimedia (T140015)]] (duration: 00m 29s) |
[production] |
| 15:09 |
<thcipriani@tin> |
Synchronized static/images/project-logos/trwikimedia.png: SWAT: [[gerrit:298441|Logo update for trwikimedia (T140015)]] (duration: 00m 33s) |
[production] |
| 14:58 |
<bblack> |
upgrading nginx to 1.11.2-1+wmf1 on cache_maps |
[production] |
| 14:33 |
<elukey> |
Rebuild new AQS Cassandra cluster (aqs100[456]) to remove previous testing settings (no prod traffic is served) |
[production] |
| 14:23 |
<reedy@tin> |
Synchronized wmf-config/extension-list: even more extension.json (duration: 00m 26s) |
[production] |
| 14:17 |
<bblack> |
nginx 1.11.2-1+wmf1 uploaded to carbon |
[production] |
| 14:17 |
<reedy@tin> |
Synchronized wmf-config/extension-list: moar extension.json (duration: 00m 26s) |
[production] |
| 14:04 |
<bblack> |
lvs nodes: apt-get install linux-meta |
[production] |
| 13:52 |
<bblack> |
lvs nodes: apt-get upgrade to latest (various base system packages) |
[production] |
| 13:49 |
<bblack> |
cache nodes: apt-get upgrade to latest (just 3.16 kernel) |
[production] |
| 13:05 |
<ottomata> |
restarting nodemanagers on analytics 1039 1046 and 1054 |
[production] |
| 10:45 |
<godog> |
terbium:~# lvextend --size +70G -r /dev/mapper/terbium--vg-root T139786 |
[production] |
| 09:35 |
<gehel> |
lowering elasticsearch codfw high watermark to rebalance cluster |
[production] |
| 09:32 |
<godog> |
reboot ms-be3004 / high load average and xfs unhappy |
[production] |
| 09:22 |
<godog> |
progressively delete esams swift containers, unused and not in production |
[production] |
| 07:49 |
<elukey> |
removing api servers mw113[0-9] from service via conftool as first decom step (T139353) |
[production] |
| 06:31 |
<legoktm@tin> |
Synchronized wmf-config/CommonSettings.php: Don't block logins if CentralAuthUser::queryAttached() fails - T119736 (duration: 00m 27s) |
[production] |
| 05:59 |
<legoktm> |
running checkLocalUsers.php on terbium |
[production] |
| 04:37 |
<ori> |
Reverted all wikis to wmf8 due to tenfold increase in T119736 |
[production] |
| 04:34 |
<ori@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: (no message) |
[production] |
| 02:26 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Tue Jul 12 02:26:41 UTC 2016 (duration 5m 31s) |
[production] |
| 02:21 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.9) (duration: 08m 44s) |
[production] |
|
2016-07-11
§
|
| 23:49 |
<maxsem@tin> |
Synchronized php-1.28.0-wmf.9/resources/: https://gerrit.wikimedia.org/r/#/c/298402/ (duration: 00m 28s) |
[production] |
| 23:36 |
<maxsem@tin> |
Synchronized php-1.28.0-wmf.9/extensions/Echo/: https://gerrit.wikimedia.org/r/#/c/298400/ (duration: 00m 33s) |
[production] |
| 23:32 |
<maxsem@tin> |
Synchronized php-1.28.0-wmf.9/extensions/Citoid/: https://gerrit.wikimedia.org/r/#/c/298327/ (duration: 00m 27s) |
[production] |
| 23:30 |
<maxsem@tin> |
Synchronized php-1.28.0-wmf.9/extensions/Wikidata: https://gerrit.wikimedia.org/r/#/c/298386/ (duration: 02m 00s) |
[production] |
| 23:15 |
<maxsem@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/297964/ (duration: 00m 32s) |
[production] |
| 23:11 |
<maxsem@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/297556/ (duration: 00m 28s) |
[production] |
| 23:07 |
<maxsem@tin> |
Synchronized php-1.28.0-wmf.9/extensions/Kartographer/: https://gerrit.wikimedia.org/r/#/c/297556/ (duration: 00m 29s) |
[production] |
| 22:46 |
<krenair@tin> |
Synchronized wmf-config: more labs-only changes: https://gerrit.wikimedia.org/r/298393 (duration: 00m 36s) |
[production] |
| 21:50 |
<krenair@tin> |
Synchronized wmf-config: sync labs-only change, should be a noop here: https://gerrit.wikimedia.org/r/#/c/298299/ (duration: 00m 39s) |
[production] |
| 21:40 |
<awight> |
resuming fundraising donations queue consumer |
[production] |
| 21:37 |
<mutante> |
ytterbium - graceful'ed Apache, warning about duplicate NameVirtual host is gone |
[production] |
| 21:03 |
<bd808> |
Updated default mapping for logstash-* index creation using json generated by https://gerrit.wikimedia.org/r/#/c/298295/. Should take effect starting with the logstash-2016.07.12 index. |
[production] |
| 20:44 |
<reedy@tin> |
Synchronized wmf-config/extension-list-labs: nooop for prod (duration: 00m 32s) |
[production] |
| 20:31 |
<ottomata> |
rolling restart of hadoop-yarn-nodemanager to apply log aggregation retention seconds |
[production] |