|
2016-02-17
§
|
| 16:09 |
<urandom> |
restbase deploy stalled at restbase1008 (under maintenance) |
[production] |
| 16:05 |
<anomie@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Namespace aliases for tawiki (task [[phab:T126604|]]) (duration: 01m 31s) |
[production] |
| 16:00 |
<urandom> |
continuing production-wide restbase deploy (15a6c50) |
[production] |
| 15:58 |
<godog> |
copy restbase.{v1_*,sys_*,ALL,GET,HEAD,POST,OPTIONS,_robots} to restbase.external on graphite1001 and graphite2001 |
[production] |
| 15:55 |
<godog> |
copy restbase.private to restbase.internal on graphite1001 and graphite2001 |
[production] |
| 15:53 |
<aude@tin> |
Synchronized php-1.27.0-wmf.13/extensions/Wikidata: Fix caching data types bug: T127095 (duration: 01m 44s) |
[production] |
| 15:53 |
<urandom> |
canary deploy of restbase to restbase1001.eqiad.wmnet (15a6c50) complete |
[production] |
| 15:53 |
<urandom> |
canary deploy of restbase to restbase1001.eqiad.wmnet (15a6c50) |
[production] |
| 15:51 |
<bblack> |
package upgrades commencing on lvs* |
[production] |
| 15:43 |
<urandom> |
restbase staging deploy (15a6c50) complete |
[production] |
| 15:38 |
<urandom> |
deploying restbase (15a6c50) in staging |
[production] |
| 15:33 |
<jynus> |
stopping puppet on all database hosts (db, dbstore, es, etc.) for lag alert testing |
[production] |
| 14:05 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1022 after maintenance (duration: 01m 34s) |
[production] |
| 14:02 |
<bblack> |
package upgrades on cp* commence |
[production] |
| 13:23 |
<elukey> |
rebooted kafka1013 for maintenance |
[production] |
| 13:06 |
<elukey> |
stopping kafka on kafka1013 |
[production] |
| 12:51 |
<moritzm> |
rolling restart of ms-fe2* |
[production] |
| 12:31 |
<moritzm> |
restarting apache on krypton for glibc update |
[production] |
| 12:00 |
<godog> |
bump stripe_cache_size to 10240 for md2 on restbase1008 |
[production] |
| 11:54 |
<jynus> |
restarting schema change on wikidatawiki (s5) T62539 |
[production] |
| 11:47 |
<moritzm> |
restarting hhvm on mw2* to put glibc update into effect |
[production] |
| 11:29 |
<jynus> |
rolling schema change on wikidatawiki (s5) |
[production] |
| 10:40 |
<jynus> |
refreshed dns server config |
[production] |
| 09:57 |
<godog> |
stop cassandra-a on restbase1008 for raid expansion |
[production] |
| 09:55 |
<godog> |
depool restbase1008 for raid expansion T119935 |
[production] |
| 08:19 |
<akosiaris> |
gnt-instance reboot alsafi.wikimedia.org |
[production] |
| 08:19 |
<akosiaris> |
enabled puppet on sca1001, sca1002 |
[production] |
| 03:15 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Wed Feb 17 03:15:40 UTC 2016 (duration 9m 30s) |
[production] |
| 03:06 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.14) (duration: 10m 31s) |
[production] |
| 02:39 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.13) (duration: 18m 59s) |
[production] |
| 01:36 |
<krenair@tin> |
Synchronized php-1.27.0-wmf.13/extensions/VisualEditor/modules/ve-mw/init/targets/ve.init.mw.DesktopArticleTarget.init.js: touch - https://phabricator.wikimedia.org/T125249#2012068 (duration: 01m 31s) |
[production] |
| 01:33 |
<krenair@tin> |
Synchronized php-1.27.0-wmf.13/extensions/VisualEditor/extension.json: https://gerrit.wikimedia.org/r/#/c/271174/ (duration: 01m 29s) |
[production] |
| 01:26 |
<mutante> |
labservices1001 - out of disk again (T126572) - moved designate-mdns.log files to /srv/var/ |
[production] |
| 01:13 |
<krenair@tin> |
Synchronized php-1.27.0-wmf.13/extensions/CentralAuth/includes/session/CentralAuthSessionProvider.php: https://gerrit.wikimedia.org/r/#/c/271153/ (duration: 01m 31s) |
[production] |
| 00:47 |
<krenair@tin> |
Synchronized php-1.27.0-wmf.13/extensions/Math/MathValidator.php: https://gerrit.wikimedia.org/r/#/c/270981/ (duration: 01m 31s) |
[production] |
| 00:33 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings-labs.php: https://gerrit.wikimedia.org/r/271074 (duration: 01m 29s) |
[production] |
| 00:29 |
<krenair@tin> |
Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/266603 (duration: 01m 30s) |
[production] |
| 00:26 |
<Krenair> |
... after showing rsync common finished line |
[production] |
| 00:25 |
<Krenair> |
sync-common taking a while to terminate |
[production] |
| 00:14 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/270985 (duration: 01m 34s) |
[production] |
|
2016-02-16
§
|
| 23:39 |
<demon@tin> |
Finished scap: pre-deploying wmf.14 to testwiki to prime l10n cache (duration: 30m 36s) |
[production] |
| 23:17 |
<hashar> |
Jenkins accepting slave creations again. Root cause is /var/lib/jenkins/config-history/nodes/ has reached the 32k inode limit. |
[production] |
| 23:12 |
<mobrovac> |
restbase deploy end of 6f69a30 |
[production] |
| 23:08 |
<demon@tin> |
Started scap: pre-deploying wmf.14 to testwiki to prime l10n cache |
[production] |
| 23:04 |
<demon@tin> |
Synchronized w/health-check.php: Ensure procinfo exists (duration: 00m 58s) |
[production] |
| 23:02 |
<hashar> |
Nodepool can not authenticate with Jenkins anymore. Thus it can not add slaves it spawned. |
[production] |
| 22:58 |
<mobrovac> |
restbase deploy start of 6f69a30 |
[production] |
| 22:56 |
<hashar> |
contint: Nodepool instances pool exhausted |
[production] |
| 22:02 |
<hashar> |
tin/mira : git -C /srv/patches/ config core.sharedRepository true |
[production] |
| 21:55 |
<chasemp> |
"chgrp -R mwdeploy /srv/mediawiki-staging/.git/objects && chmod -R g+w /srv/mediawiki-staging/.git/objects" on tin |
[production] |