|
2016-05-30
§
|
| 12:35 |
<gehel> |
re-enabling puppet on elasticsearch codfw cluster (T133125) |
[production] |
| 12:07 |
<gehel> |
nginx restarted on elasticsearch codfw cluster (T133125) |
[production] |
| 11:53 |
<godog> |
cherry-pick https://gerrit.wikimedia.org/r/#/c/280652 https://gerrit.wikimedia.org/r/#/c/290479 https://gerrit.wikimedia.org/r/#/c/291710/ on tools-puppetmaster-01 |
[tools] |
| 11:42 |
<moritzm> |
upgrading hhvm in codfw (also picking up updated versions of icu and lcms) |
[production] |
| 10:57 |
<moritzm> |
upgrading hhvm on remaining canaries (also picking up updated versions of icu and lcms) |
[production] |
| 10:35 |
<hashar> |
Restarted Zuul. |
[production] |
| 10:31 |
<moritzm> |
upgrading hhvm on mw1017 (also picking up updated versions of icu and lcms) |
[production] |
| 10:30 |
<hashar> |
Zuul deadlocked :( |
[production] |
| 10:12 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Reduce db1071 load (duration: 00m 48s) |
[production] |
| 09:57 |
<moritzm> |
installing libidn security updates on jessie systems |
[production] |
| 09:09 |
<gehel> |
shutting down elasticsearch on codfw for upgrade (T133125) |
[production] |
| 08:44 |
<volans> |
Align thread_pool_max_threads to my.cnf value on 1 slave/shard in eqiad (db1065,db1076,db1078,db1040,db1026,db1061,db1039) T133333 |
[production] |
| 08:27 |
<gehel> |
starting elasticsearch upgrade on codfw (T133125) |
[production] |
| 08:04 |
<ori> |
zuul is back up but jobs which were enqueued are gone |
[releng] |
| 07:50 |
<ori> |
restarting jenkins on gallium, too |
[releng] |
| 07:49 |
<ori> |
restarted zuul-merger service on gallium |
[releng] |
| 07:44 |
<ori> |
Disconnecting and then reconnecting Gearman from Jenkins did not appear to do anything; going to depool / repool nodes. |
[releng] |
| 07:42 |
<ori> |
Temporarily disconnecting Gearman from Jenkins, per <https://www.mediawiki.org/wiki/Continuous_integration/Zuul#Known_issues> |
[releng] |
| 02:30 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Mon May 30 02:30:46 UTC 2016 (duration 5m 52s) |
[production] |
| 02:24 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.3) (duration: 09m 04s) |
[production] |
|
2016-05-29
§
|
| 18:58 |
<YuviPanda> |
deleted tools-k8s-bastion-01 for T136496 |
[tools] |
| 17:36 |
<halfak_> |
pushed wikilabels-wikimedia-config:6ca744c to labels production |
[wikilabels] |
| 17:29 |
<halfak_> |
pushed wikilabels-wikimedia-config:6ca744c to labels-staging |
[wikilabels] |
| 17:17 |
<halfak_> |
pushed wikilabels-wikimedia-config:3b0ba53 to labels-staging |
[wikilabels] |
| 16:57 |
<halfak_> |
pushed wikilabels-wikimedia-config:6bfc83e to labels production |
[wikilabels] |
| 16:48 |
<halfak_> |
pushed wikilabels-wikimedia-config:6bfc83e to labels-staging |
[wikilabels] |
| 16:37 |
<halfak_> |
pushed wikilabels-wikimedia-config:9eabc81 to production |
[wikilabels] |
| 16:35 |
<halfak_> |
pushed wikilabels-wikimedia-config:9eabc81 to labels-staging |
[wikilabels] |
| 16:28 |
<halfak_> |
pushed wikilabels-wikimedia-config:3a3a850 to labels-staging |
[wikilabels] |
| 16:01 |
<halfak_> |
pushed wikilabels-wikimedia-config:3f7b958 to labels-staging |
[wikilabels] |
| 14:29 |
<valhallasw`cloud> |
chowned /data/project/xtools-mab-dev to root and back to stop rogue process that was writing to the directory. I'm still not sure where that process was running, but at least this seems to have solved the issue |
[tools] |
| 02:58 |
<urandom> |
Bootsrapping restbase2004-b.codfw.wmet : T134016 |
[production] |
| 02:27 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Sun May 29 02:27:27 UTC 2016 (duration 5m 27s) |
[production] |
| 02:22 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.3) (duration: 08m 35s) |
[production] |
|
2016-05-28
§
|
| 22:25 |
<Amir1> |
deploy d13dd3c into prod |
[wikilabels] |
| 22:14 |
<Amir1> |
deploying 6065701 to prod |
[wikilabels] |
| 21:51 |
<valhallasw`cloud> |
rebooted tools-webgrid-lighttpd-1408, tools-pastion-01, tools-exec-1205 |
[tools] |
| 21:21 |
<valhallasw`cloud> |
rebooting tools-exec-1204 (T136495) |
[tools] |
| 19:47 |
<hoo> |
Updated Wikidata's property suggester with data from Monday's json dump and removed the external identifiers as a workaround for T132839 |
[production] |
| 13:59 |
<urandom> |
Bootstrapping restbase1011-c.eqiad.wmnet : T134016 |
[production] |
| 04:43 |
<ori> |
depooling integration-slave-trusty-1015 to profile phpunit runs |
[releng] |
| 02:26 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Sat May 28 02:26:10 UTC 2016 (duration 5m 47s) |
[production] |
| 02:20 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.3) (duration: 08m 20s) |
[production] |