|
2016-05-30
§
|
| 10:30 |
<hashar> |
Zuul deadlocked :( |
[production] |
| 10:12 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Reduce db1071 load (duration: 00m 48s) |
[production] |
| 09:57 |
<moritzm> |
installing libidn security updates on jessie systems |
[production] |
| 09:09 |
<gehel> |
shutting down elasticsearch on codfw for upgrade (T133125) |
[production] |
| 08:44 |
<volans> |
Align thread_pool_max_threads to my.cnf value on 1 slave/shard in eqiad (db1065,db1076,db1078,db1040,db1026,db1061,db1039) T133333 |
[production] |
| 08:27 |
<gehel> |
starting elasticsearch upgrade on codfw (T133125) |
[production] |
| 08:04 |
<ori> |
zuul is back up but jobs which were enqueued are gone |
[releng] |
| 07:50 |
<ori> |
restarting jenkins on gallium, too |
[releng] |
| 07:49 |
<ori> |
restarted zuul-merger service on gallium |
[releng] |
| 07:44 |
<ori> |
Disconnecting and then reconnecting Gearman from Jenkins did not appear to do anything; going to depool / repool nodes. |
[releng] |
| 07:42 |
<ori> |
Temporarily disconnecting Gearman from Jenkins, per <https://www.mediawiki.org/wiki/Continuous_integration/Zuul#Known_issues> |
[releng] |
| 02:30 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Mon May 30 02:30:46 UTC 2016 (duration 5m 52s) |
[production] |
| 02:24 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.3) (duration: 09m 04s) |
[production] |
|
2016-05-29
§
|
| 18:58 |
<YuviPanda> |
deleted tools-k8s-bastion-01 for T136496 |
[tools] |
| 17:36 |
<halfak_> |
pushed wikilabels-wikimedia-config:6ca744c to labels production |
[wikilabels] |
| 17:29 |
<halfak_> |
pushed wikilabels-wikimedia-config:6ca744c to labels-staging |
[wikilabels] |
| 17:17 |
<halfak_> |
pushed wikilabels-wikimedia-config:3b0ba53 to labels-staging |
[wikilabels] |
| 16:57 |
<halfak_> |
pushed wikilabels-wikimedia-config:6bfc83e to labels production |
[wikilabels] |
| 16:48 |
<halfak_> |
pushed wikilabels-wikimedia-config:6bfc83e to labels-staging |
[wikilabels] |
| 16:37 |
<halfak_> |
pushed wikilabels-wikimedia-config:9eabc81 to production |
[wikilabels] |
| 16:35 |
<halfak_> |
pushed wikilabels-wikimedia-config:9eabc81 to labels-staging |
[wikilabels] |
| 16:28 |
<halfak_> |
pushed wikilabels-wikimedia-config:3a3a850 to labels-staging |
[wikilabels] |
| 16:01 |
<halfak_> |
pushed wikilabels-wikimedia-config:3f7b958 to labels-staging |
[wikilabels] |
| 14:29 |
<valhallasw`cloud> |
chowned /data/project/xtools-mab-dev to root and back to stop rogue process that was writing to the directory. I'm still not sure where that process was running, but at least this seems to have solved the issue |
[tools] |
| 02:58 |
<urandom> |
Bootsrapping restbase2004-b.codfw.wmet : T134016 |
[production] |
| 02:27 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Sun May 29 02:27:27 UTC 2016 (duration 5m 27s) |
[production] |
| 02:22 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.3) (duration: 08m 35s) |
[production] |
|
2016-05-28
§
|
| 22:25 |
<Amir1> |
deploy d13dd3c into prod |
[wikilabels] |
| 22:14 |
<Amir1> |
deploying 6065701 to prod |
[wikilabels] |
| 21:51 |
<valhallasw`cloud> |
rebooted tools-webgrid-lighttpd-1408, tools-pastion-01, tools-exec-1205 |
[tools] |
| 21:21 |
<valhallasw`cloud> |
rebooting tools-exec-1204 (T136495) |
[tools] |
| 19:47 |
<hoo> |
Updated Wikidata's property suggester with data from Monday's json dump and removed the external identifiers as a workaround for T132839 |
[production] |
| 13:59 |
<urandom> |
Bootstrapping restbase1011-c.eqiad.wmnet : T134016 |
[production] |
| 04:43 |
<ori> |
depooling integration-slave-trusty-1015 to profile phpunit runs |
[releng] |
| 02:26 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Sat May 28 02:26:10 UTC 2016 (duration 5m 47s) |
[production] |
| 02:20 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.3) (duration: 08m 20s) |
[production] |
|
2016-05-27
§
|
| 23:58 |
<ori> |
Forcing a Puppet run on logstash* |
[production] |
| 23:44 |
<legoktm> |
ran extensions/GlobalBlocking/fixGlobalBlockWhitelist.php for T56496 |
[production] |
| 23:15 |
<legoktm@tin> |
Synchronized php-1.28.0-wmf.3/includes/actions/RollbackAction.php: RollbackAction: Don't return true, causes '1' to be output (duration: 00m 34s) |
[production] |
| 22:50 |
<legoktm> |
mwscript initSiteStats.php --wiki=lrcwiki --update for T109635 |
[production] |
| 21:52 |
<MaxSem> |
Rebooted maps-tiles1, maps-tiles2 and maps-warper due to NFS troubles |
[maps] |
| 19:49 |
<krinkle@tin> |
Synchronized php-1.28.0-wmf.3/includes/Linker.php: Iba17ce55ff9 (duration: 00m 25s) |
[production] |
| 19:49 |
<krinkle@tin> |
Synchronized php-1.28.0-wmf.3/includes/actions/RollbackAction.php: Iba17ce55ff9 (duration: 00m 31s) |
[production] |
| 19:29 |
<hasharAway> |
Refreshed Nodepool images |
[releng] |
| 19:06 |
<paravoid> |
un-draining esams |
[production] |
| 18:57 |
<valhallasw`cloud> |
sudo qconf -Ae /var/lib/gridengine/etc/exechosts/toolsbeta-exec-1209.toolsbeta.eqiad.wmflabs |
[toolsbeta] |
| 18:15 |
<thcipriani> |
restarted zuul due to deadlock issue |
[production] |
| 18:13 |
<thcipriani> |
restarting zuul for deadlock |
[releng] |
| 18:00 |
<thcipriani> |
Reloading Zuul to deploy I0c3aeacf92d430ad1272f5f00e7fb7182b8a05bf |
[releng] |
| 17:10 |
<volans> |
Stop slave, stop mysql and shutdown es2017 and es2019 for hardware maintenance T130702 |
[production] |