|
2016-02-17
§
|
| 15:53 |
<urandom> |
canary deploy of restbase to restbase1001.eqiad.wmnet (15a6c50) |
[production] |
| 15:51 |
<bblack> |
package upgrades commencing on lvs* |
[production] |
| 15:43 |
<urandom> |
restbase staging deploy (15a6c50) complete |
[production] |
| 15:38 |
<urandom> |
deploying restbase (15a6c50) in staging |
[production] |
| 15:33 |
<jynus> |
stopping puppet on all database hosts (db, dbstore, es, etc.) for lag alert testing |
[production] |
| 14:05 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1022 after maintenance (duration: 01m 34s) |
[production] |
| 14:02 |
<bblack> |
package upgrades on cp* commence |
[production] |
| 13:23 |
<elukey> |
rebooted kafka1013 for maintenance |
[production] |
| 13:06 |
<elukey> |
stopping kafka on kafka1013 |
[production] |
| 12:51 |
<moritzm> |
rolling restart of ms-fe2* |
[production] |
| 12:31 |
<moritzm> |
restarting apache on krypton for glibc update |
[production] |
| 12:00 |
<godog> |
bump stripe_cache_size to 10240 for md2 on restbase1008 |
[production] |
| 11:54 |
<jynus> |
restarting schema change on wikidatawiki (s5) T62539 |
[production] |
| 11:47 |
<moritzm> |
restarting hhvm on mw2* to put glibc update into effect |
[production] |
| 11:29 |
<jynus> |
rolling schema change on wikidatawiki (s5) |
[production] |
| 10:40 |
<jynus> |
refreshed dns server config |
[production] |
| 09:57 |
<godog> |
stop cassandra-a on restbase1008 for raid expansion |
[production] |
| 09:55 |
<godog> |
depool restbase1008 for raid expansion T119935 |
[production] |
| 08:19 |
<akosiaris> |
gnt-instance reboot alsafi.wikimedia.org |
[production] |
| 08:19 |
<akosiaris> |
enabled puppet on sca1001, sca1002 |
[production] |
| 03:15 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Wed Feb 17 03:15:40 UTC 2016 (duration 9m 30s) |
[production] |
| 03:06 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.14) (duration: 10m 31s) |
[production] |
| 02:39 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.13) (duration: 18m 59s) |
[production] |
| 01:36 |
<krenair@tin> |
Synchronized php-1.27.0-wmf.13/extensions/VisualEditor/modules/ve-mw/init/targets/ve.init.mw.DesktopArticleTarget.init.js: touch - https://phabricator.wikimedia.org/T125249#2012068 (duration: 01m 31s) |
[production] |
| 01:33 |
<krenair@tin> |
Synchronized php-1.27.0-wmf.13/extensions/VisualEditor/extension.json: https://gerrit.wikimedia.org/r/#/c/271174/ (duration: 01m 29s) |
[production] |
| 01:26 |
<mutante> |
labservices1001 - out of disk again (T126572) - moved designate-mdns.log files to /srv/var/ |
[production] |
| 01:13 |
<krenair@tin> |
Synchronized php-1.27.0-wmf.13/extensions/CentralAuth/includes/session/CentralAuthSessionProvider.php: https://gerrit.wikimedia.org/r/#/c/271153/ (duration: 01m 31s) |
[production] |
| 00:47 |
<krenair@tin> |
Synchronized php-1.27.0-wmf.13/extensions/Math/MathValidator.php: https://gerrit.wikimedia.org/r/#/c/270981/ (duration: 01m 31s) |
[production] |
| 00:33 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings-labs.php: https://gerrit.wikimedia.org/r/271074 (duration: 01m 29s) |
[production] |
| 00:29 |
<krenair@tin> |
Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/266603 (duration: 01m 30s) |
[production] |
| 00:26 |
<Krenair> |
... after showing rsync common finished line |
[production] |
| 00:25 |
<Krenair> |
sync-common taking a while to terminate |
[production] |
| 00:14 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/270985 (duration: 01m 34s) |
[production] |
|
2016-02-16
§
|
| 23:39 |
<demon@tin> |
Finished scap: pre-deploying wmf.14 to testwiki to prime l10n cache (duration: 30m 36s) |
[production] |
| 23:17 |
<hashar> |
Jenkins accepting slave creations again. Root cause is /var/lib/jenkins/config-history/nodes/ has reached the 32k inode limit. |
[production] |
| 23:12 |
<mobrovac> |
restbase deploy end of 6f69a30 |
[production] |
| 23:08 |
<demon@tin> |
Started scap: pre-deploying wmf.14 to testwiki to prime l10n cache |
[production] |
| 23:04 |
<demon@tin> |
Synchronized w/health-check.php: Ensure procinfo exists (duration: 00m 58s) |
[production] |
| 23:02 |
<hashar> |
Nodepool can not authenticate with Jenkins anymore. Thus it can not add slaves it spawned. |
[production] |
| 22:58 |
<mobrovac> |
restbase deploy start of 6f69a30 |
[production] |
| 22:56 |
<hashar> |
contint: Nodepool instances pool exhausted |
[production] |
| 22:02 |
<hashar> |
tin/mira : git -C /srv/patches/ config core.sharedRepository true |
[production] |
| 21:55 |
<chasemp> |
"chgrp -R mwdeploy /srv/mediawiki-staging/.git/objects && chmod -R g+w /srv/mediawiki-staging/.git/objects" on tin |
[production] |
| 20:50 |
<demon@tin> |
Finished scap: removing expired wmf.9 branch (duration: 28m 03s) |
[production] |
| 20:22 |
<demon@tin> |
Started scap: removing expired wmf.9 branch |
[production] |
| 20:14 |
<twentyafterfour> |
restarted phd on iridium to kick-start the outbound email queue |
[production] |
| 19:38 |
<hashar@tin> |
Synchronized multiversion/updateBranchPointers: Missing require_once MWWikiversions (duration: 00m 57s) |
[production] |
| 19:16 |
<ori@mira> |
Synchronized wmf-config/InitialiseSettings.php: I2926f73b78fa0: Revert "Test HTML stripping in production mobile beta" (duration: 01m 29s) |
[production] |
| 19:01 |
<hashar> |
tin: checking out mw 1.27.0-wmf.14 |
[production] |
| 18:15 |
<ori@tin> |
Synchronized wmf-config/mobile.php: I5818f0350925: Don't serve HiDPI thumbs on mobile web (duration: 00m 58s) |
[production] |