|
2018-02-13
§
|
| 10:51 |
<filippo@tin> |
Synchronized wmf-config/ProductionServices.php: depool poolcounter1002 for disk replacement (duration: 00m 56s) |
[production] |
| 10:49 |
<moritzm> |
shutdown deployment-tmh01 (T187063) |
[releng] |
| 10:28 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1099 (duration: 00m 54s) |
[production] |
| 10:21 |
<Hauskatze> |
purged old IP data from abuse_filter_log |
[releng] |
| 10:08 |
<godog> |
roll-restart ms-fe in codfw/eqiad after applying https://gerrit.wikimedia.org/r/c/409942/ |
[production] |
| 10:03 |
<ema> |
restart pybal on lvs2003 |
[production] |
| 09:58 |
<ema> |
restart pybal on lvs2006 |
[production] |
| 09:52 |
<filippo@neodymium> |
conftool action : set/pooled=no; selector: name=ms-fe2005.codfw.wmnet |
[production] |
| 09:49 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2075, depool db2038 and db2059 (duration: 00m 55s) |
[production] |
| 09:32 |
<marostegui> |
Stop mysql on db2075 for mysql and kernel upgrade |
[production] |
| 09:30 |
<marostegui> |
Stop replication in sync on db1089 and dbstore1002 - T162807 |
[production] |
| 09:29 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1065 - T162807 (duration: 00m 54s) |
[production] |
| 09:22 |
<elukey> |
powercycle analytics1062 - not reachable via ssh, frozen via serial console |
[production] |
| 09:22 |
<jynus> |
disabling puppet on all eqiad databases |
[production] |
| 09:20 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1065 - T162807 (duration: 00m 55s) |
[production] |
| 09:20 |
<marostegui> |
Stop replication in sync on db1089 and db1065 - T162807 |
[production] |
| 09:15 |
<hashar> |
beta: scap fixed by deleting a leftover /srv/mediawiki/.~tmp~/ |
[releng] |
| 09:12 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2084:3315, depool db2075 (duration: 00m 55s) |
[production] |
| 09:12 |
<hashar> |
deployment-jobrunner02: sudo rm -fR /srv/mediawiki/.~tmp~/ # might be causing the beta-scap-eqiad Jenkins job to fail |
[releng] |
| 09:01 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1099:3311 - T162807 (duration: 00m 55s) |
[production] |
| 08:52 |
<marostegui> |
Stop replication in sync on db1089 and db1099:3311 - T162807 |
[production] |
| 08:51 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1089, db1099 - T162807 (duration: 00m 55s) |
[production] |
| 08:41 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Depool db2084:3315 (duration: 00m 56s) |
[production] |
| 08:37 |
<hashar> |
tin.eqiad.wmnet: removing live hack in /srv/mediawiki-staging/scap/plugins/clean.py | T187160 |
[production] |
| 08:32 |
<moritzm> |
installing wavpack security updates |
[production] |
| 08:09 |
<moritzm> |
installing exim security updates on trusty hosts |
[production] |
| 07:02 |
<marostegui> |
Deploy schema change on s5 db2089 db2084 db2075 db2039 db2059 - T187089 |
[production] |
| 06:28 |
<marostegui> |
reload haproxy on dbproxy1005 |
[production] |
| 05:10 |
<bblack@neodymium> |
conftool action : set/pooled=yes; selector: name=cp50(0[12345789]|1[12]).eqsin.wmnet |
[production] |
| 03:44 |
<legoktm> |
deployed https://gerrit.wikimedia.org/r/410100 |
[releng] |
| 02:34 |
<l10nupdate@tin> |
scap sync-l10n completed (1.31.0-wmf.20) (duration: 05m 29s) |
[production] |
| 01:52 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/410089 |
[releng] |
| 00:24 |
<cwd> |
re-enabled p-c |
[production] |
| 00:16 |
<demon@tin> |
Synchronized php-1.31.0-wmf.20/extensions/VisualEditor/modules/ve-mw/ui/pages/: T187112 (duration: 00m 56s) |
[production] |
| 00:10 |
<cwd> |
disabled p-c jobs for reboot |
[production] |
| 00:04 |
<demon@tin> |
Synchronized wmf-config/: cleanup Sentry inclusion for labs, should be no-op (duration: 00m 57s) |
[production] |
| 00:03 |
<demon@tin> |
Synchronized wmf-config/InitialiseSettings.php: cleanup Sentry inclusion for labs, should be no-op (duration: 00m 56s) |
[production] |
|
2018-02-12
§
|
| 23:47 |
<demon@tin> |
Finished deploy [gerrit/gerrit@6adde70]: reviewers plugin (duration: 00m 12s) |
[production] |
| 23:46 |
<demon@tin> |
Started deploy [gerrit/gerrit@6adde70]: reviewers plugin |
[production] |
| 23:32 |
<mutante> |
terbium,wasat: touch /var/log/mediawwiki/purge_abusefilter.log ; set owner/permissions like other logfiles |
[production] |
| 23:16 |
<elukey> |
re-run webrequest-load-wf-upload-2018-2-12-21 via Hue (node managers failure) |
[analytics] |
| 23:13 |
<elukey> |
manual restart of Yarn Node Managers on analytics1058/31 (failed due to root partition filled up for the issue logged before) |
[production] |
| 23:13 |
<elukey> |
manual restart of Yarn Node Managers on analytics1058/31 |
[analytics] |
| 23:09 |
<elukey> |
cleaned up tmp files on all analytics hadoop worker nodes, job filling up tmp |
[analytics] |
| 23:09 |
<elukey> |
cleaned up tmp files on all analytics hadoop worker nodes, job filling up tmp |
[production] |
| 23:06 |
<legoktm> |
deployed https://gerrit.wikimedia.org/r/410075 https://gerrit.wikimedia.org/r/410076 |
[releng] |
| 22:07 |
<andrewbogott> |
opened up port 5668 to labweb1001 and 1002 in the 'proxy' security group |
[project-proxy] |
| 21:44 |
<Sagan> |
Neon: Space on / was low (only 2 GB left), run apt clean, freed 5,8 GB of space |
[rcm] |
| 21:27 |
<andrew@tin> |
Finished deploy [horizon/deploy@2f70002]: updating several submodules, probably breaking static content (duration: 03m 18s) |
[production] |
| 21:24 |
<andrew@tin> |
Started deploy [horizon/deploy@2f70002]: updating several submodules, probably breaking static content |
[production] |