|
2017-12-21
§
|
| 19:25 |
<robh> |
cp4032 going offline for memory swap |
[production] |
| 17:24 |
<jynus> |
starting manual backup of x1-master onto db1055 |
[production] |
| 17:01 |
<volans> |
debugging Icinga notes_url (no side effect expected but logging it in case there will be) T170353 |
[production] |
| 16:56 |
<volans> |
restarted ircecho to pick up gerrit/399658 |
[production] |
| 15:43 |
<joal@tin> |
Finished deploy [analytics/refinery@92f9318]: Deploying for new pageview_top_bycountry job (duration: 04m 40s) |
[production] |
| 15:39 |
<joal@tin> |
Started deploy [analytics/refinery@92f9318]: Deploying for new pageview_top_bycountry job |
[production] |
| 12:39 |
<elukey> |
restart druid historical/broker to apply new jvm settings to druid public workers (druid100[456] - https://gerrit.wikimedia.org/r/399617) |
[production] |
| 12:07 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=mw1333.*.eqiad.wmnet |
[production] |
| 11:43 |
<_joe_> |
refreshing puppet facts on the puppet compilers |
[production] |
| 10:54 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Remove db1055 & db1056 (duration: 00m 51s) |
[production] |
| 10:52 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Remove db1055 & db1056 (duration: 00m 51s) |
[production] |
| 10:30 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Restore db1082 and db1100 original weight - T161294 (duration: 00m 51s) |
[production] |
| 10:15 |
<elukey> |
rolling restart of eventbus on kafka* for openssl security updates |
[production] |
| 10:03 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase traffic for db1100 and start restoring original weight for db1082 - T161294 (duration: 00m 48s) |
[production] |
| 09:50 |
<elukey> |
restart eventbus on kafka2001 for openssl updates |
[production] |
| 09:39 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase traffic for db1100 - T161294 (duration: 00m 51s) |
[production] |
| 09:30 |
<elukey> |
restart zookeeper on conf100[2,3] for jvm updates - T179943 |
[production] |
| 09:22 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1100 - T161294 (duration: 00m 51s) |
[production] |
| 09:20 |
<elukey> |
restart zookeeper on conf1001 for jvm updates - T179943 |
[production] |
| 08:51 |
<elukey> |
run kafka preferred-replica-election after maintenance of kafka1023 (fully bootstrapped now) |
[production] |
| 08:35 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1100 - T161294 (duration: 00m 51s) |
[production] |
| 08:34 |
<marostegui> |
Stop replication in sync on db1100 and dbstore1002 - T161294 |
[production] |
| 08:23 |
<elukey> |
repool mw1277 after investigation |
[production] |
| 07:37 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Slowly repool db1100- T161294 (duration: 00m 52s) |
[production] |
| 07:21 |
<marostegui> |
Upgrade MariaDB on db1100 |
[production] |
| 07:16 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1109 and db1096:3315 - T161294 (duration: 00m 51s) |
[production] |
| 06:55 |
<_joe_> |
restarting hhvm on mw1231 and mw1208 |
[production] |
| 06:50 |
<marostegui> |
Stop replication in sync on dbstore1002 - db1100 - T161294 |
[production] |
| 06:42 |
<marostegui> |
Stop replication in sync on db1100 - db2052 - T161294 |
[production] |
| 06:36 |
<marostegui> |
Remove some old files in dbstore1001:/srv/tmp to address the WARNING alert |
[production] |
| 06:35 |
<marostegui> |
Stop replication in sync db1100 and db1071 - T161294 |
[production] |
| 06:33 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1109 - T161294 (duration: 00m 51s) |
[production] |
| 03:59 |
<mutante> |
mw1315 kill and restart hhvm | mw1312 stop and start hhvm |
[production] |
| 03:55 |
<mutante> |
mw1290 kill and restart hhvm | mw1230 stop and start hhvm |
[production] |
| 02:20 |
<l10nupdate@tin> |
scap sync-l10n completed (1.31.0-wmf.12) (duration: 05m 31s) |
[production] |
| 01:16 |
<eileen> |
update CiviCRM civicrm revision changed from 9d6ba74f57 to ffa9d7fc7a, config revision is da6fa4cce9 |
[production] |
|
2017-12-20
§
|
| 23:26 |
<twentyafterfour> |
restarting apache on phab1001 to deploy a hotfix for T144184 |
[production] |
| 21:42 |
<mepps> |
updated dash from 114131713e to b01458b260 |
[production] |
| 21:20 |
<otto@tin> |
Finished deploy [analytics/refinery@548dad7]: deploying refinery v0.0.56 with JsonRefine fixes to allow Popups schema to be refined. This is a no-op for everything else (duration: 04m 51s) |
[production] |
| 21:15 |
<otto@tin> |
Started deploy [analytics/refinery@548dad7]: deploying refinery v0.0.56 with JsonRefine fixes to allow Popups schema to be refined. This is a no-op for everything else |
[production] |
| 20:31 |
<ebernhardson> |
T183053 update elasticsearch settings for wikidatawiki_content on codfw to use: index.refresh_interval=5s |
[production] |
| 20:18 |
<XioNoX> |
remove local-as from cr2-esams IX4 |
[production] |
| 19:46 |
<XioNoX> |
remove local-as from cr2-esams IX6 |
[production] |
| 17:57 |
<_joe_> |
depooling mw1277 for further investigation |
[production] |
| 17:51 |
<demon@tin> |
Synchronized wmf-config/InitialiseSettings.php: T172875 (second try, forgot to pull first (duration: 00m 51s) |
[production] |
| 17:48 |
<elukey> |
new mw jobrunner in production (mw1334) - T165519 |
[production] |