|
2015-06-29
§
|
| 15:18 |
<krenair> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/221047/ (duration: 00m 13s) |
[production] |
| 15:12 |
<krenair> |
Synchronized php-1.26wmf11/extensions/ContentTranslation/modules/tools/ext.cx.tools.link.js: https://gerrit.wikimedia.org/r/#/c/221605 (duration: 00m 13s) |
[production] |
| 15:02 |
<krenair> |
Synchronized php-1.26wmf11/extensions/ContentTranslation/modules/tools/ext.cx.tools.formatter.js: https://gerrit.wikimedia.org/r/#/c/221604/ (duration: 00m 14s) |
[production] |
| 14:34 |
<jynus> |
rebooting and reinstalling db1022 |
[production] |
| 13:17 |
<dcausse> |
restarting Elasticsearch to pick up new plugin versions |
[releng] |
| 12:06 |
<YuviPanda> |
restarting rsync with new exclusions file on labstore1002 to codfw |
[production] |
| 12:06 |
<YuviPanda> |
excluded maps, mwoffliner and video project from rsync of broken FS to speed it up |
[production] |
| 11:59 |
<YuviPanda> |
interupt rsync on labstore1001 to prevent it from copying mwofflienr files |
[production] |
| 11:00 |
<_joe_> |
shutting down etcd1003, cleaning exported resources |
[production] |
| 10:43 |
<hashar> |
manually rebasing integration-puppetmaster git repo |
[releng] |
| 10:32 |
<_joe_> |
effectively removing etcd1003 from the cluster |
[production] |
| 10:24 |
<hashar> |
restarted puppetmater on deployment-salt |
[releng] |
| 10:23 |
<hashar> |
puppet master stalled due to: [ldap-yaml-enc.p] <defunct> . Killing it |
[releng] |
| 10:21 |
<hashar> |
sees beta cluster puppetmaster is suffering from some random issue |
[releng] |
| 10:17 |
<_joe_> |
starting removal of etcd1003 from the etcd cluster |
[production] |
| 08:49 |
<_joe_> |
joined conf1003 to the etcd cluster |
[production] |
| 08:21 |
<jynus> |
Synchronized wmf-config/db-eqiad.php: Depool db1022 for reinstall (duration: 00m 12s) |
[production] |
| 08:12 |
<_joe_> |
adding conf1002 to the etcd cluster as a member |
[production] |
| 07:46 |
<akosiaris> |
disabling ntp everywhere expect selected hosts in anticipation for the leap second |
[production] |
| 04:51 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Mon Jun 29 04:51:48 UTC 2015 (duration 51m 47s) |
[production] |
| 03:08 |
<jgage> |
jmxtrans filled disks on all kafka brokers, 21GB log files. removed logs and restarted services. |
[production] |
| 02:23 |
<LocalisationUpdate> |
completed (1.26wmf11) at 2015-06-29 02:23:47+00:00 |
[production] |
| 02:20 |
<l10nupdate> |
Synchronized php-1.26wmf11/cache/l10n: (no message) (duration: 05m 53s) |
[production] |
| 00:52 |
<springle> |
restart eventlogging auto-purge on m4 |
[production] |
| 00:51 |
<springle> |
restart replication on dbstore2002 |
[production] |
| 00:00 |
<springle> |
pausing replication on dbstore2002 |
[production] |
|
2015-06-27
§
|
| 23:30 |
<bd808> |
Deleted corrupt shards on logstash1004 and logstash1005. Recovery in process |
[production] |
| 20:12 |
<ori> |
Delegated full access to Google Webmaster Tools for myself (olivneh@). |
[production] |
| 04:58 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Sat Jun 27 04:58:46 UTC 2015 (duration 58m 45s) |
[production] |
| 02:42 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/221343 & https://gerrit.wikimedia.org/r/221344 |
[releng] |
| 02:36 |
<Krinkle> |
Reloading Zuul to deploy https://gerrit.wikimedia.org/r/221342 |
[releng] |
| 02:23 |
<LocalisationUpdate> |
completed (1.26wmf11) at 2015-06-27 02:23:40+00:00 |
[production] |
| 02:22 |
<Krinkle> |
Reloading Zuul to deploy https://gerrit.wikimedia.org/r/221338 |
[releng] |
| 02:20 |
<l10nupdate> |
Synchronized php-1.26wmf11/cache/l10n: (no message) (duration: 05m 46s) |
[production] |
| 02:15 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/#/c/221337/ |
[releng] |
| 01:56 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/221333 & https://gerrit.wikimedia.org/r/221334 |
[releng] |
| 01:42 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/221331 |
[releng] |
| 01:36 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/221330 |
[releng] |
| 01:28 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/221329 |
[releng] |
| 01:13 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/221328 |
[releng] |
| 00:15 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/221316 & https://gerrit.wikimedia.org/r/221318 |
[releng] |