751-800 of 10000 results (28ms)
2017-06-16 §
09:13 <moritzm> re-enabled puppet on mw2129 (no reason was given why it was disabled( [production]
08:50 <jynus> bringing down pc1005 and pc1006 for maintenance T167567 [production]
08:40 <jynus@tin> Synchronized wmf-config/db-codfw.php: Add db1099 and db1001 hosts to config (duration: 00m 41s) [production]
08:23 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Switchover pc1005 and pc1006 to db1099 and db1001 (duration: 00m 45s) [production]
08:20 <jynus> about to swithover pc1005 and pc1006 to db1099 and db1001 [production]
05:45 <ebernhardson> increase enwiki_content replicas on codfw from 2 to 3 to match eqiad [production]
02:37 <l10nupdate@tin> ResourceLoader cache refresh completed at Fri Jun 16 02:37:05 UTC 2017 (duration 6m 25s) [production]
02:30 <l10nupdate@tin> scap sync-l10n completed (1.30.0-wmf.5) (duration: 07m 02s) [production]
2017-06-15 §
23:37 <mutante> added stretch support for jenkins (https://gerrit.wikimedia.org/r/#/c/359227/, https://gerrit.wikimedia.org/r/#/c/359356/) | 'reprepro copy stretch-wikimedia jessie-wikimedia jenkins' to make .deb available on stretch | releases1001 now running jenkins , icinga recovered | (hashar) (T164030) [production]
23:30 <mutante> APT - reprepro copy strech-wikimedia jessie-wikimedia jenkins (copy existing jenkins package to stretch, it can be used on both) [production]
23:18 <ebernhardson@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: T166408: Remove dead config variable MinervaPrintStyles (duration: 00m 41s) [production]
23:15 <ebernhardson@tin> Finished scap: wmf-config Scap: T162276: Enable crossproject search (duration: 03m 37s) [production]
23:11 <ebernhardson@tin> Started scap: wmf-config Scap: T162276: Enable crossproject search [production]
23:10 <ebernhardson@tin> Synchronized wmf-config/InitialiseSettings.php: Scap: T162276: Enable crossproject search (duration: 00m 51s) [production]
22:59 <mutante> mw2251 - repooled [production]
22:56 <mutante> mw2251 - scap pull [production]
22:53 <ebernhardson> restarting elasticsearch on relforge to pickup new ltr-query plugin [production]
22:30 <ejegg> updated DjangoBannerStats from 9e6b11712851919e4318930e19bdf9c941d8ae38 to 5963e7c116f083ada28476c1c41ff1b46232d772 [production]
22:02 <volans> restarted pdfrender on scb1001 T159922 [production]
21:45 <mutante> powercycling mw2251 (frozen console) [production]
21:39 <volans@puppetmaster1001> conftool action : set/pooled=inactive; selector: name=mw2251.codfw.wmnet [production]
21:37 <volans> re-enabled puppet and force run to re-enable ircecho on einstenium [production]
21:29 <demon@tin> Finished scap: Removing Cards extension (duration: 21m 49s) [production]
21:08 <demon@tin> Started scap: Removing Cards extension [production]
20:57 <mutante> upgrading RT (request tracker) [production]
19:35 <demon@tin> rebuilt wikiversions.php and synchronized wikiversions files: group2 to wmf.5 [production]
19:22 <ladsgroup@tin> Finished deploy [ores/deploy@ab88a74]: Deploying gerrit:359224/1 for missing config variables (duration: 24m 15s) [production]
19:17 <XioNoX> Re-enabled link between cr2-codfw and cr1-eqdfw - T167261 [production]
18:58 <ladsgroup@tin> Started deploy [ores/deploy@ab88a74]: Deploying gerrit:359224/1 for missing config variables [production]
18:44 <paravoid> restarting all puppetmasters [production]
18:40 <paravoid> temporarily stopping icinga-wm [production]
18:27 <demon@tin> Synchronized wmf-config/CirrusSearch-common.php: Remove quirks and enable token_count_router thingie (duration: 00m 44s) [production]
18:16 <demon@tin> Synchronized php-1.30.0-wmf.5/includes/libs/objectcache/MultiWriteBagOStuff.php: T167465 (duration: 00m 44s) [production]
18:14 <demon@tin> Synchronized wmf-config/InitialiseSettings.php: T167617 (duration: 00m 44s) [production]
18:12 <demon@tin> Synchronized wmf-config/FeaturedFeedsWMF.php: T167617 (duration: 00m 44s) [production]
17:50 <mutante> install2002 - re-enabled puppet, reverted live hack, back to normal (issue seems to be NIC or other) [production]
17:28 <mutante> install2002 - temp disabling puppet and applying hot fix to debug install issue for papaul [production]
17:27 <bblack> disabling puppet on cp*wmnet to avoid puppet races on https://gerrit.wikimedia.org/r/#/c/341729 merge [production]
14:39 <gehel> killing stuck replication on maps1001 [production]
14:38 <krinkle@tin> Synchronized wmf-config/CommonSettings.php: no-op Ifc7b1ea80 - Remove EtcdConfig from beta (duration: 00m 45s) [production]
13:24 <gehel> elasticsearch upgrade to 5.3.2 on relforge cluster completed, cluster still recovering - T163708 [production]
13:23 <aude@tin> Synchronized wmf-config/Wikibase.php: Add constraints statements section on Wikidata T167126 (duration: 00m 43s) [production]
13:19 <dcausse> [cirrus] reindexing all zh wikis (eqiad & codfw) [production]
13:14 <aude@tin> Synchronized wmf-config/InitialiseSettings.php: Enable BM25 for Chinese wikis (duration: 00m 44s) [production]
13:13 <aude@tin> Synchronized tests/cirrusTest.php: (no justification provided) (duration: 00m 45s) [production]
13:02 <gehel> starting elasticsearch upgrade to 5.3.2 on relforge cluster - T163708 [production]
12:14 <gehel> restart elasticsearch on relforge1001 to validate latest config changes [production]
10:16 <moritzm> rollout remaining systemd updates from jessie point release [production]
09:14 <jynus> shutting down and deleting data at pc1004 for cloning from db1096 [production]
09:10 <hashar> Jenkins back up and happy. [production]