|
2015-01-14
§
|
| 19:38 |
<reedy> |
Started scap: testwiki to 1.25wmf15 and build l10n caches. Some extension bumps in wmf14 |
[production] |
| 18:19 |
<mutante> |
deleted /etc/logrotate.d/dumpwikidatajson on snapshot1003 (gerrit 182173) |
[production] |
| 17:30 |
<Reedy> |
mw1152: Permission denied (publickey). |
[production] |
| 17:29 |
<reedy> |
Purged l10n cache for 1.25wmf12 |
[production] |
| 17:27 |
<Reedy> |
deleted php-1.25wmf5 through php-1.25wmf9 from /srv/mediawiki |
[production] |
| 17:27 |
<YuviPanda> |
sync-common && scap-rebuild-cdbs completed on virt1000, wikitech still seems up. Tending to the wounded. |
[production] |
| 17:23 |
<YuviPanda> |
rm -rf /srv/mediawiki-staging/php-1.25wmf5 on tin |
[production] |
| 16:57 |
<YuviPanda> |
running sync-common —verbose on virt1000 |
[production] |
| 16:55 |
<krenair> |
Synchronized wmf-config/wikitech.php: https://gerrit.wikimedia.org/r/#/c/184635/ (duration: 00m 06s) |
[production] |
| 16:48 |
<krenair> |
Synchronized php-1.25wmf14/extensions/MultimediaViewer/resources/mmv/mmv.lightboxinterface.js: https://gerrit.wikimedia.org/r/#/c/184633/ (duration: 00m 07s) |
[production] |
| 16:38 |
<krenair> |
Synchronized php-1.25wmf14: https://gerrit.wikimedia.org/r/#/c/184818/ (duration: 06m 14s) |
[production] |
| 15:46 |
<ottomata> |
stopping all varnishkafka bits instances |
[production] |
| 10:15 |
<ori> |
Synchronized wmf-config/mc.php: typo fix for nutcracker socket (duration: 00m 05s) |
[production] |
| 10:05 |
<ori> |
Synchronized wmf-config/mc.php: nutcracker: use UNIX domain socket only if on HHVM (duration: 00m 07s) |
[production] |
| 09:57 |
<ori> |
Synchronized wmf-config/mc.php: Memcached: make remaining app servers use UNIX domain socket (duration: 00m 06s) |
[production] |
| 08:24 |
<_joe_> |
repooling mw1225, depooled for a long time |
[production] |
| 07:44 |
<ori> |
Synchronized wmf-config/mc.php: use UNIX domain socket on mw12* app servers (duration: 00m 06s) |
[production] |
| 04:15 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Wed Jan 14 04:15:41 UTC 2015 (duration 15m 40s) |
[production] |
| 02:24 |
<LocalisationUpdate> |
completed (1.25wmf14) at 2015-01-14 02:24:36+00:00 |
[production] |
| 02:24 |
<l10nupdate> |
Synchronized php-1.25wmf14/cache/l10n: (no message) (duration: 00m 01s) |
[production] |
| 02:16 |
<LocalisationUpdate> |
completed (1.25wmf13) at 2015-01-14 02:16:18+00:00 |
[production] |
| 02:16 |
<l10nupdate> |
Synchronized php-1.25wmf13/cache/l10n: (no message) (duration: 00m 01s) |
[production] |
| 01:17 |
<maxsem> |
Synchronized wmf-config: https://gerrit.wikimedia.org/r/184812 (duration: 00m 06s) |
[production] |
| 01:01 |
<maxsem> |
Synchronized wmf-config: touch (duration: 00m 08s) |
[production] |
| 00:59 |
<maxsem> |
Synchronized wmf-config: touch (duration: 00m 07s) |
[production] |
| 00:50 |
<maxsem> |
Synchronized wmf-config/: https://gerrit.wikimedia.org/r/184810 https://gerrit.wikimedia.org/r/184757 (duration: 00m 06s) |
[production] |
| 00:46 |
<maxsem> |
Synchronized php-1.25wmf14/extensions/Wikidata/: https://gerrit.wikimedia.org/r/#/c/184807/ (duration: 00m 11s) |
[production] |
| 00:28 |
<maxsem> |
Synchronized php-1.25wmf14/extensions/Wikidata/: https://gerrit.wikimedia.org/r/#/c/184807/ (duration: 00m 12s) |
[production] |
|
2015-01-13
§
|
| 23:50 |
<ori> |
Updated nutcracker on application servers to 0.4.0+dfsg-1+wm1. |
[production] |
| 22:52 |
<ori> |
Synchronized wmf-config/mc.php: Use UNIX domain socket for nutcracker on mw1030 & mw1031 (duration: 00m 05s) |
[production] |
| 22:37 |
<hashar> |
Restarted Zuul, deadlocked waiting for Gerrit |
[production] |
| 22:37 |
<ejegg> |
updated payments from c23cf16407ef200da446d81fb990abbe429fd378 to 46bd1611c56d6e31594d01aa345c35dd45dcf676 |
[production] |
| 20:48 |
<Reedy> |
running mwscript extensions/CirrusSearch/maintenance/updateSearchIndexConfig.php --wiki=mediawikiwiki --reindexAndRemoveOk --indexIdentifier=now |
[production] |
| 20:48 |
<reedy> |
Synchronized wmf-config/InitialiseSettings.php: mediawikiwiki content namespaces (duration: 00m 05s) |
[production] |
| 20:42 |
<reedy> |
Synchronized wmf-config/: Config updates (duration: 00m 06s) |
[production] |
| 20:41 |
<reedy> |
Synchronized database lists: wikidata dblist update (duration: 00m 06s) |
[production] |
| 19:38 |
<reedy> |
Synchronized wmf-config/InitialiseSettings.php: Add Renameuser debug log group (duration: 00m 09s) |
[production] |
| 19:24 |
<reedy> |
Synchronized wmf-config/Wikibase.php: bump cache epoch (duration: 00m 06s) |
[production] |
| 19:09 |
<reedy> |
Synchronized wmf-config/InitialiseSettings.php: monolog: enable for group0 + group1 wikis (duration: 00m 07s) |
[production] |
| 19:06 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Non Wikipedias to 1.25wmf14 |
[production] |
| 18:40 |
<ori> |
mw1062: sync-file failed, read-only file system. Host should be removed from dsh group. |
[production] |
| 18:38 |
<ori> |
Synchronized wmf-config/StartProfiler.php: xenon: Skip frames that don't have a 'phpStack' key (duration: 00m 06s) |
[production] |
| 17:39 |
<hashar> |
Zuul back in action. Got recheck or +2 again the changes that have been discarded. |
[production] |
| 17:37 |
<hashar> |
Restarting deadlocked Zuul , which drops ALL events. Reason is Gerrit lost connection with its database which is not handled by Zuul . See https://wikitech.wikimedia.org/wiki/Incident_documentation/20150106-Zuul |
[production] |
| 16:12 |
<demon> |
Synchronized wmf-config/: (no message) (duration: 00m 05s) |
[production] |
| 16:12 |
<demon> |
Synchronized flaggedrevs.dblist: (no message) (duration: 00m 05s) |
[production] |
| 16:02 |
<YuviPanda> |
restart mysql on virt1000, wikitech acting up again |
[production] |
| 15:23 |
<godog> |
upgrade and restart poolcounter on helium |
[production] |
| 15:17 |
<godog> |
upload poolcounter 1.0.3 to precise-wikimedia |
[production] |
| 15:17 |
<YuviPanda> |
restarted mysql on virt1000, wikitech failing with db errors. seems fine now (4 minutes ago) |
[production] |