|
2015-03-04
§
|
| 13:11 |
<bblack> |
depooled amssq31 in esams for reinstall |
[production] |
| 07:24 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Wed Mar 4 07:23:48 UTC 2015 (duration 23m 47s) |
[production] |
| 02:29 |
<LocalisationUpdate> |
completed (1.25wmf19) at 2015-03-04 02:28:15+00:00 |
[production] |
| 02:28 |
<l10nupdate> |
Synchronized php-1.25wmf19/cache/l10n: (no message) (duration: 00m 01s) |
[production] |
| 02:16 |
<LocalisationUpdate> |
completed (1.25wmf18) at 2015-03-04 02:15:10+00:00 |
[production] |
| 02:15 |
<l10nupdate> |
Synchronized php-1.25wmf18/cache/l10n: (no message) (duration: 00m 01s) |
[production] |
| 00:44 |
<demon> |
Finished scap: evening swat: centralauth, VE, user.php fix (duration: 30m 19s) |
[production] |
| 00:14 |
<demon> |
Started scap: evening swat: centralauth, VE, user.php fix |
[production] |
| 00:09 |
<gwicke> |
running concurrent test dumps of enwiki and dewiki through xenon |
[production] |
|
2015-03-03
§
|
| 23:39 |
<mutante> |
ran puppet on ruthenium (keeps showing up in icinga but then no issue when you run it) |
[production] |
| 22:56 |
<ori> |
adding four additional txstatsd backends to graphite1001 to cope with load |
[production] |
| 22:00 |
<hashar> |
reboot integration-puppetmaster in case it solves a NFS mount issue |
[releng] |
| 21:49 |
<demon> |
Synchronized docroot/mediawiki/keys/: (no message) (duration: 00m 05s) |
[production] |
| 21:45 |
<gwicke> |
re-initializing cassandra on test hosts xenon, praseodymium and cerium for new test run; expect some downtime |
[production] |
| 20:33 |
<legoktm> |
manually created centralauth.users_to_rename table |
[releng] |
| 20:05 |
<bblack> |
cp10(60|64|65|70),amssq42 repooled in pybal |
[production] |
| 19:30 |
<demon> |
Synchronized multiversion/MWMultiVersion.php: moar debugging (duration: 00m 07s) |
[production] |
| 19:30 |
<twentyafterfour> |
rebuilt wikiversions.cdb and synchronized wikiversions files: group1 to 1.25wmf19 |
[production] |
| 19:05 |
<twentyafterfour> |
Starting the tuesday train deploy ( Group1 wikis to 1.25wmf19 ) |
[production] |
| 18:48 |
<Krinkle> |
New builds run fine, but there's 30 stuck builds occupying executors. Their output is finished and Zuul/Gerrit got the event already, but they won't die. |
[releng] |
| 18:39 |
<bblack> |
depooled cp10(60|65|70),amssq42 temporarily in pybal for reinstalls |
[production] |
| 18:30 |
<Krinkle> |
The stuck builds have a line of "Finished: .." in them, but are still showing a loader spinner on the bottom of their build log 10 minutes later. |
[releng] |
| 18:28 |
<Krinkle> |
Lots of Jenkins builds are stuck half-way executing. No clear cause. Everything seems up. |
[releng] |
| 17:18 |
<Krinkle> |
Reloading Zuul to deploy Icad0a26dc8 and Icac172b16 |
[releng] |
| 17:03 |
<marktraceur> |
Synchronized wmf-config/InitialiseSettings.php: [SWAT] [config] Enable BounceHandler on non-wikipedias (duration: 00m 05s) |
[production] |
| 16:53 |
<marktraceur> |
Synchronized wmf-config/InitialiseSettings.php: [SWAT] [config] Remove redundant namespace aliases for Nepali Wikipedia (newiki) (duration: 00m 05s) |
[production] |
| 16:41 |
<marktraceur> |
Synchronized php-1.25wmf19/extensions/Graph/: [SWAT] [wmf19] Update Graph to 25.19 (duration: 00m 07s) |
[production] |
| 16:30 |
<marktraceur> |
Synchronized wmf-config/InitialiseSettings.php: [SWAT] [config] Add namespace aliases for Nepali Wikipedia (newiki) (duration: 00m 08s) |
[production] |
| 16:22 |
<marktraceur> |
Synchronized wmf-config/InitialiseSettings-labs.php: [SWAT] [config] Kartik forgot to sync this beta config patch for CX (duration: 00m 10s) |
[production] |
| 16:22 |
<marktraceur> |
Synchronized wmf-config/CommonSettings-labs.php: [SWAT] [config] Enable Flickr uploads on betacommons (duration: 00m 06s) |
[production] |
| 15:39 |
<hashar> |
cancelled logrotate update of all jobs since that seems to kill the Jenkins/Zuul gearman connection. Probably because all jobs are registered on each config change. |
[releng] |
| 15:31 |
<hashar> |
updating all jobs in Jenkins based on PS2 of https://gerrit.wikimedia.org/r/194109 |
[releng] |
| 10:56 |
<hashar> |
Created instance i-000008fb with image "ubuntu-14.04-trusty" and hostname i-000008fb.eqiad.wmflabs. |
[releng] |
| 10:52 |
<hashar> |
deleting integration-puppetmaster to recreate it with a new image {bug|T87484} . Will have to reapply I5335ea7cbfba33e84b3ddc6e3dd83a7232b8acfd and I30e5bfeac398e0f88e538c75554439fe82fcc1cf |
[releng] |
| 03:47 |
<Krinkle> |
git-deploy: Deploying integration/slave-scripts 05a5593..1e64ed9 |
[releng] |
| 03:31 |
<springle> |
codfw replag coming up, schema changes |
[production] |
| 02:29 |
<andyrussg> |
Synchronized php-1.25wmf18/extensions/CentralNotice/: CenralNotice update (duration: 00m 07s) |
[production] |
| 02:18 |
<andyrussg> |
Synchronized php-1.25wmf19/extensions/CentralNotice/: CenralNotice update (duration: 00m 09s) |
[production] |
| 02:17 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Tue Mar 3 02:16:47 UTC 2015 (duration 16m 46s) |
[production] |
| 02:06 |
<MaxSem> |
Creating geo_tags table everywhere it's not yet present |
[production] |
| 02:06 |
<LocalisationUpdate> |
completed (1.25wmf19) at 2015-03-03 02:04:56+00:00 |
[production] |
| 02:04 |
<l10nupdate> |
Synchronized php-1.25wmf19/cache/l10n: (no message) (duration: 00m 01s) |
[production] |
| 02:04 |
<LocalisationUpdate> |
completed (1.25wmf18) at 2015-03-03 02:03:26+00:00 |
[production] |
| 02:03 |
<l10nupdate> |
Synchronized php-1.25wmf18/cache/l10n: (no message) (duration: 00m 02s) |
[production] |
| 01:11 |
<marxarelli> |
gzip'd /var/log/account/pacct.0 on deployment-bastion to free space |
[releng] |
| 00:52 |
<ejegg> |
enabled queue consumers |
[production] |
| 00:42 |
<ejegg> |
updated crm from 3c002f32e04652ae56a4fe791bc6158ab981ed8d to f8fb0f61531431348f3a8a3ee107056a864d537b |
[production] |
| 00:39 |
<ejegg> |
disabled queue consumers |
[production] |
| 00:23 |
<demon> |
Synchronized php-1.25wmf18/extensions/Flow: (no message) (duration: 00m 07s) |
[production] |
| 00:15 |
<demon> |
Synchronized wmf-config/: cors logging for beta (duration: 00m 06s) |
[production] |