|
2014-08-26
§
|
| 19:58 |
<bd808> |
Ran sync-common on mw1053.eqiad.wmnet to recover from failure during last scap |
[production] |
| 19:48 |
<aude> |
Finished scap: Update new messages for Wikibase (duration: 07m 16s) |
[production] |
| 19:41 |
<aude> |
Started scap: Update new messages for Wikibase |
[production] |
| 19:39 |
<aude> |
Synchronized wmf-config/Wikibase.php: add Wikibase badges css setting (duration: 00m 10s) |
[production] |
| 19:26 |
<aude> |
Synchronized wmf-config/Wikibase.php: enable new serialization format for wikidata (duration: 00m 08s) |
[production] |
| 19:10 |
<reedy> |
Synchronized php-1.24wmf18/extensions/Echo/: (no message) (duration: 00m 14s) |
[production] |
| 19:05 |
<aude> |
Synchronized wmf-config/Wikibase.php: enable otherprojects sidebar beta feature (duration: 00m 15s) |
[production] |
| 18:55 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Non wikipedias to 1.24wmf18 |
[production] |
| 18:53 |
<reedy> |
Synchronized php-1.24wmf18/extensions/MassMessage: (no message) (duration: 00m 14s) |
[production] |
| 18:53 |
<reedy> |
Synchronized php-1.24wmf17/extensions/MassMessage: (no message) (duration: 00m 16s) |
[production] |
| 18:19 |
<jgage> |
Failover from analytics1010-eqiad-wmnet to analytics1004-eqiad-wmnet successful |
[production] |
| 17:47 |
<bd808> |
Synchronized private/PrivateSettings.php: Syncing file rather than symlink (duration: 00m 04s) |
[production] |
| 17:36 |
<bd808> |
mw1010.eqiad.wmnet was out of sync too. I suspect there is something wrong with the fanout update step in scap |
[production] |
| 17:26 |
<bd808> |
/usr/local/apache/common-local out of date on mw1161.eqiad.wmnet; updated via sync-common |
[production] |
| 17:25 |
<bd808> |
sync-* not updating terbium properly; sync-common from terbium manually got several config changes; maybe a problem with mw1161.eqiad.wmnet rsync mirror |
[production] |
| 17:14 |
<demon> |
Synchronized wmf-config/InitialiseSettings.php: touch (duration: 00m 04s) |
[production] |
| 17:12 |
<demon> |
Synchronized wmf-config/PrivateSettings.php: adjust swift auth url for cirrus (duration: 00m 04s) |
[production] |
| 17:05 |
<cmjohnson> |
swapping failed disk labsdb1003 slot 1 |
[production] |
| 16:42 |
<bd808> |
Ran sync-common on osmium to verify that it now rebuilds l10n cache by default (and it does!) |
[production] |
| 16:36 |
<legoktm> |
running removeOldManualUserPages.php (GlobalCssJs) for users who requested it |
[production] |
| 16:29 |
<demon> |
Synchronized wmf-config/InitialiseSettings.php: Again, with feeling (duration: 00m 04s) |
[production] |
| 16:26 |
<bd808> |
Finished scap: no-op scap to test scap code update (duration: 13m 31s) |
[production] |
| 16:20 |
<bd808|DEPLOY> |
Rsync sloooow to fenari "16:18:52 fenari INFO - Finished rsync common (duration: 04m 38s)" |
[production] |
| 16:12 |
<bd808> |
Started scap: no-op scap to test scap code update |
[production] |
| 16:07 |
<demon> |
Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 04s) |
[production] |
| 16:07 |
<bd808|DEPLOY> |
Updated scap to 116027f (Make sync-common update l10n cdb files by default) |
[production] |
| 15:06 |
<anomie> |
Synchronized wmf-config: SWAT: Enable GlobalCssJs on all CentralAuth wikis minus loginwiki [[gerrit:154432]] (duration: 00m 09s) |
[production] |
| 13:33 |
<hashar> |
Jenkins mediawiki-core-qunit job has been switched to Zuul cloner and pass! :-D |
[production] |
| 13:29 |
<_joe_> |
re-enabling puppet, change aborted as not all sites are served via hhvm on the hhvm appservers (true story). Will re-do once all configs are in their place |
[production] |
| 13:12 |
<_joe_> |
disabling puppet on all appservers while deploying an apache change |
[production] |
| 12:48 |
<springle> |
Synchronized wmf-config/db-eqiad.php: db1054 to normal load (duration: 00m 06s) |
[production] |
| 12:33 |
<hashar> |
Jenkins reverted mediawiki-core-qunit to use Zuul cloner {{gerrit|156268}}. Gotta play with it on a new job name since it does not work out of the box as expected. |
[production] |
| 12:12 |
<hashar> |
Jenkins migrating mediawiki-core-qunit to use Zuul cloner {{gerrit|156268}} |
[production] |
| 12:03 |
<akosiaris> |
disable puppet on labsdb1006 for planet osm import |
[production] |
| 11:53 |
<springle> |
Synchronized wmf-config/db-eqiad.php: pool db1054, warm up (duration: 00m 08s) |
[production] |
| 09:04 |
<godog> |
reboot ms-be1011, unresponse on network and console |
[production] |
| 08:28 |
<springle> |
Synchronized wmf-config/db-eqiad.php: repool db1036 (duration: 00m 06s) |
[production] |
| 05:41 |
<springle> |
xtrabackup clone db1036 to db1054 |
[production] |
| 05:39 |
<springle> |
Synchronized wmf-config/db-eqiad.php: depool db1036 while cloning (duration: 00m 06s) |
[production] |
| 05:28 |
<springle> |
upgrade & restart db1054, fs check |
[production] |
| 04:48 |
<demon> |
Synchronized wmf-config/InitialiseSettings-labs.php: (no message) (duration: 00m 06s) |
[production] |
| 04:27 |
<springle> |
labsdb1002 back up |
[production] |
| 04:07 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Tue Aug 26 04:06:34 UTC 2014 (duration 6m 33s) |
[production] |
| 03:23 |
<^d> |
restarting elasticsearch on elastic1001, elastic1003 and elastic1008. icinga may complain briefly. |
[production] |
| 03:11 |
<springle> |
filesystem issues on labsdb1002. stopped mysqld |
[production] |
| 03:05 |
<LocalisationUpdate> |
completed (1.24wmf18) at 2014-08-26 03:04:18+00:00 |
[production] |
| 02:34 |
<LocalisationUpdate> |
completed (1.24wmf17) at 2014-08-26 02:33:00+00:00 |
[production] |