|
2016-11-21
§
|
| 12:07 |
<elukey> |
restarted oozie bundles via hue after oozie/hive restart |
[analytics] |
| 11:56 |
<elukey> |
restarted jobchron/runner on mw208[0-5] since systemd was reporting degradation (broken pipes in the journald logs) |
[production] |
| 11:45 |
<elukey> |
stopped all the oozie bundles via Hue as prep-step for Hive/Oozie daemons restarts on an1003 |
[analytics] |
| 11:32 |
<gehel> |
starting cluster restart on elasticsearch eqiad for JVM upgrade |
[production] |
| 10:29 |
<jynus> |
performing blocking schema change on db2065 T151029 |
[production] |
| 10:15 |
<elukey> |
created 0039892-161020124223818-oozie-oozi-C and 0039895-161020124223818-oozie-oozi-C for webrequest-load-wf-(text|upload)-2016-11-21-8 |
[analytics] |
| 10:11 |
<UltimateKarate> |
blah blah blah |
[production] |
| 10:11 |
<marostegui> |
Deploy ALTER table db1068 commonswiki.revision - https://phabricator.wikimedia.org/T147305 |
[production] |
| 10:05 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1068 - T147305 (duration: 00m 48s) |
[production] |
| 09:49 |
<ema> |
removed varnishrls.service from non-cache_text hosts |
[production] |
| 08:50 |
<elukey> |
rolling restart of hadoop-related java daemons on analytics* hosts due to openjdk update |
[production] |
| 08:08 |
<marostegui> |
Deploy ALTER table db1069 commonswiki.revision - https://phabricator.wikimedia.org/T147305 |
[production] |
| 07:18 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2070 - T149553 (duration: 00m 49s) |
[production] |
| 06:11 |
<legoktm@tin> |
Synchronized wmf-config/: Disable centralauth-rename right for maintenance (T148242, T151155) (duration: 00m 52s) |
[production] |
| 02:21 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Mon Nov 21 02:21:35 UTC 2016 (duration 4m 19s) |
[production] |
| 02:17 |
<l10nupdate@tin> |
scap sync-l10n completed (1.29.0-wmf.3) (duration: 05m 46s) |
[production] |
| 00:52 |
<bd808> |
Cherry-picked https://gerrit.wikimedia.org/r/#/c/322605 to /data/project/admin/toollabs |
[tools.admin] |
|
2016-11-19
§
|
| 22:31 |
<paladox> |
deployed https://gerrit.wikimedia.org/r/#/c/322487/ |
[tools.lolrrit-wm] |
| 22:20 |
<paladox> |
deploying https://gerrit.wikimedia.org/r/#/c/320294/ |
[tools.lolrrit-wm] |
| 22:16 |
<Zppix> |
merging https://gerrit.wikimedia.org/r/#/c/320294/ |
[tools.lolrrit-wm] |
| 02:20 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Sat Nov 19 02:20:37 UTC 2016 (duration 4m 18s) |
[production] |
| 02:16 |
<l10nupdate@tin> |
scap sync-l10n completed (1.29.0-wmf.3) (duration: 05m 27s) |
[production] |
| 01:54 |
<Krinkle> |
[horizon.wikimedia.org] Retiring cvn-app5 (removing public IP, shutting down instance). To be deleted in a few days. |
[cvn] |
| 01:49 |
<Krinkle> |
Restored CVNBot20 database from cvn-app5 (retired) to cvn-app7 (got corrupted in transfer somehow) |
[cvn] |
| 01:43 |
<Krinkle> |
CVNBot20 database backup 2 days ago (Nov 16) also corrupt. |
[cvn] |
| 01:40 |
<Krinkle> |
CVNBot20 database was corrupted for unknown reasons. Replaced with a copy of CVNBot14's database. |
[cvn] |
| 00:15 |
<Krinkle> |
'sudo chmod 660 */CVNBot.ini' in /srv/cvn/services/cvnbot on cvn-app6 and cvn-app7 |
[cvn] |
| 00:10 |
<Krinkle> |
Reloading Zuul to deploy https://gerrit.wikimedia.org/r/322370 |
[releng] |
| 00:09 |
<Krinkle> |
'sudo chmod 775 */CVNBot.exe */Lists.sqlite' in /srv/cvn/services/cvnbot on cvn-app6 and cvn-app7 |
[cvn] |
|
2016-11-18
§
|
| 23:06 |
<rxy_> |
rxy@cvn-app7 "sudo chmod g+w -R /srv/cvn/git/stillalive" for write permission of group members |
[cvn] |
| 23:05 |
<rxy_> |
rxy@cvn-app6 "sudo chmod g+w -R /srv/cvn/git/stillalive" for write permission of group members |
[cvn] |
| 22:56 |
<rxy_> |
rxy@cvn-app6 "sudo chmod g+w /srv/cvn/services/cvnbot" for write permission of group members |
[cvn] |
| 22:56 |
<rxy_> |
rxy@cvn-app6 "sudo chgrp project-cvn /srv/cvn/services/cvnbot" for write permission of group members |
[cvn] |
| 22:49 |
<rxy_> |
rxy@cvn-app7 "sudo chmod g+w /srv/cvn/services/cvnbot" for write permission of group members |
[cvn] |
| 22:48 |
<rxy_> |
rxy@cvn-app7 "sudo chgrp project-cvn /srv/cvn/services/cvnbot" for write permission of group members |
[cvn] |
| 21:32 |
<hashar> |
CI / Zuul slightly overloaded. Will resolve by itself soon. |
[production] |
| 18:52 |
<demon@tin> |
Synchronized wmf-config/CommonSettings.php: extdist settings for 1.28 (duration: 00m 49s) |
[production] |
| 18:27 |
<jynus> |
deployed unix_auth on silver (labswiki) T150446 |
[production] |
| 18:22 |
<jynus> |
removing mysql-test dir from silver to free up some space there |
[production] |
| 17:34 |
<ori@tin> |
Synchronized wmf-config/InitialiseSettings.php: If3b80b1a: Revert "Don't use AbuseFilterCachingParser on bgwiki" (T148660) (duration: 00m 50s) |
[production] |
| 16:48 |
<ejegg> |
disabled job 'Dedupe CiviCRM contacts (name-match)' |
[production] |
| 16:46 |
<ejegg> |
disabled job 'Dedupe CiviCRM contacts' |
[production] |
| 16:23 |
<Krenair> |
re-enabling and running puppet |
[shinken] |