|
2017-06-15
ยง
|
| 21:45 |
<mutante> |
powercycling mw2251 (frozen console) |
[production] |
| 21:39 |
<volans@puppetmaster1001> |
conftool action : set/pooled=inactive; selector: name=mw2251.codfw.wmnet |
[production] |
| 21:37 |
<volans> |
re-enabled puppet and force run to re-enable ircecho on einstenium |
[production] |
| 21:29 |
<demon@tin> |
Finished scap: Removing Cards extension (duration: 21m 49s) |
[production] |
| 21:08 |
<demon@tin> |
Started scap: Removing Cards extension |
[production] |
| 20:57 |
<mutante> |
upgrading RT (request tracker) |
[production] |
| 19:35 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group2 to wmf.5 |
[production] |
| 19:22 |
<ladsgroup@tin> |
Finished deploy [ores/deploy@ab88a74]: Deploying gerrit:359224/1 for missing config variables (duration: 24m 15s) |
[production] |
| 19:17 |
<XioNoX> |
Re-enabled link between cr2-codfw and cr1-eqdfw - T167261 |
[production] |
| 18:58 |
<ladsgroup@tin> |
Started deploy [ores/deploy@ab88a74]: Deploying gerrit:359224/1 for missing config variables |
[production] |
| 18:44 |
<paravoid> |
restarting all puppetmasters |
[production] |
| 18:40 |
<paravoid> |
temporarily stopping icinga-wm |
[production] |
| 18:27 |
<demon@tin> |
Synchronized wmf-config/CirrusSearch-common.php: Remove quirks and enable token_count_router thingie (duration: 00m 44s) |
[production] |
| 18:16 |
<demon@tin> |
Synchronized php-1.30.0-wmf.5/includes/libs/objectcache/MultiWriteBagOStuff.php: T167465 (duration: 00m 44s) |
[production] |
| 18:14 |
<demon@tin> |
Synchronized wmf-config/InitialiseSettings.php: T167617 (duration: 00m 44s) |
[production] |
| 18:12 |
<demon@tin> |
Synchronized wmf-config/FeaturedFeedsWMF.php: T167617 (duration: 00m 44s) |
[production] |
| 17:50 |
<mutante> |
install2002 - re-enabled puppet, reverted live hack, back to normal (issue seems to be NIC or other) |
[production] |
| 17:28 |
<mutante> |
install2002 - temp disabling puppet and applying hot fix to debug install issue for papaul |
[production] |
| 17:27 |
<bblack> |
disabling puppet on cp*wmnet to avoid puppet races on https://gerrit.wikimedia.org/r/#/c/341729 merge |
[production] |
| 14:39 |
<gehel> |
killing stuck replication on maps1001 |
[production] |
| 14:38 |
<krinkle@tin> |
Synchronized wmf-config/CommonSettings.php: no-op Ifc7b1ea80 - Remove EtcdConfig from beta (duration: 00m 45s) |
[production] |
| 13:24 |
<gehel> |
elasticsearch upgrade to 5.3.2 on relforge cluster completed, cluster still recovering - T163708 |
[production] |
| 13:23 |
<aude@tin> |
Synchronized wmf-config/Wikibase.php: Add constraints statements section on Wikidata T167126 (duration: 00m 43s) |
[production] |
| 13:19 |
<dcausse> |
[cirrus] reindexing all zh wikis (eqiad & codfw) |
[production] |
| 13:14 |
<aude@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable BM25 for Chinese wikis (duration: 00m 44s) |
[production] |
| 13:13 |
<aude@tin> |
Synchronized tests/cirrusTest.php: (no justification provided) (duration: 00m 45s) |
[production] |
| 13:02 |
<gehel> |
starting elasticsearch upgrade to 5.3.2 on relforge cluster - T163708 |
[production] |
| 12:14 |
<gehel> |
restart elasticsearch on relforge1001 to validate latest config changes |
[production] |
| 10:16 |
<moritzm> |
rollout remaining systemd updates from jessie point release |
[production] |
| 09:14 |
<jynus> |
shutting down and deleting data at pc1004 for cloning from db1096 |
[production] |
| 09:10 |
<hashar> |
Jenkins back up and happy. |
[production] |
| 09:05 |
<moritzm> |
reenable puppet on notebook1002, was disabled for the merge of the zookeeper role refactor two days ago, can be re-enabled now |
[production] |
| 09:04 |
<hashar> |
Restarting Jenkins. It seems I managed to deadlock it |
[production] |
| 08:52 |
<ariel@tin> |
Finished deploy [dumps/dumps@1734c6d]: history dump rebalance script, fixup for extension script dumps, root logger for misc dumps (duration: 00m 02s) |
[production] |
| 08:52 |
<ariel@tin> |
Started deploy [dumps/dumps@1734c6d]: history dump rebalance script, fixup for extension script dumps, root logger for misc dumps |
[production] |
| 08:40 |
<gehel> |
restart relforge1001 to validate latest config changes |
[production] |
| 08:16 |
<akosiaris@tin> |
Finished deploy [citoid/deploy@ba0db9c]: Remove the bad PMCID test from spec (duration: 07m 44s) |
[production] |
| 08:09 |
<akosiaris@tin> |
Started deploy [citoid/deploy@ba0db9c]: Remove the bad PMCID test from spec |
[production] |
| 08:02 |
<moritzm> |
updating HHVM on terbium/wasat to 3.18 |
[production] |
| 07:57 |
<akosiaris@tin> |
Finished deploy [citoid/deploy@ba0db9c]: Remove the bad PMCID test from spec (duration: 00m 38s) |
[production] |
| 07:56 |
<akosiaris@tin> |
Started deploy [citoid/deploy@ba0db9c]: Remove the bad PMCID test from spec |
[production] |
| 07:48 |
<akosiaris> |
schedule 2 hours downtime for all citoid endpoints health on scb boxes |
[production] |
| 06:08 |
<marostegui> |
Deploy alter table s2 - labsdb1003 - T166205 |
[production] |
| 05:50 |
<marostegui> |
Deploy alter table s2 - db1018 - T166205 |
[production] |
| 05:49 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Add comments to db1018 current status - T166205 (duration: 00m 43s) |
[production] |
| 05:41 |
<marostegui> |
Deploy alter table s4 - dbstore1001 - T166206 |
[production] |
| 05:22 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1036 - T166205 (duration: 00m 44s) |
[production] |
| 02:50 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Thu Jun 15 02:50:16 UTC 2017 (duration 6m 48s) |
[production] |
| 02:43 |
<l10nupdate@tin> |
scap sync-l10n completed (1.30.0-wmf.5) (duration: 07m 34s) |
[production] |
| 02:26 |
<l10nupdate@tin> |
scap sync-l10n completed (1.30.0-wmf.4) (duration: 09m 15s) |
[production] |