1651-1700 of 10000 results (44ms)
2017-06-15 §
17:28 <mutante> install2002 - temp disabling puppet and applying hot fix to debug install issue for papaul [production]
17:27 <bblack> disabling puppet on cp*wmnet to avoid puppet races on https://gerrit.wikimedia.org/r/#/c/341729 merge [production]
14:39 <gehel> killing stuck replication on maps1001 [production]
14:38 <krinkle@tin> Synchronized wmf-config/CommonSettings.php: no-op Ifc7b1ea80 - Remove EtcdConfig from beta (duration: 00m 45s) [production]
13:24 <gehel> elasticsearch upgrade to 5.3.2 on relforge cluster completed, cluster still recovering - T163708 [production]
13:23 <aude@tin> Synchronized wmf-config/Wikibase.php: Add constraints statements section on Wikidata T167126 (duration: 00m 43s) [production]
13:19 <dcausse> [cirrus] reindexing all zh wikis (eqiad & codfw) [production]
13:14 <aude@tin> Synchronized wmf-config/InitialiseSettings.php: Enable BM25 for Chinese wikis (duration: 00m 44s) [production]
13:13 <aude@tin> Synchronized tests/cirrusTest.php: (no justification provided) (duration: 00m 45s) [production]
13:02 <gehel> starting elasticsearch upgrade to 5.3.2 on relforge cluster - T163708 [production]
12:14 <gehel> restart elasticsearch on relforge1001 to validate latest config changes [production]
10:16 <moritzm> rollout remaining systemd updates from jessie point release [production]
09:14 <jynus> shutting down and deleting data at pc1004 for cloning from db1096 [production]
09:10 <hashar> Jenkins back up and happy. [production]
09:05 <moritzm> reenable puppet on notebook1002, was disabled for the merge of the zookeeper role refactor two days ago, can be re-enabled now [production]
09:04 <hashar> Restarting Jenkins. It seems I managed to deadlock it [production]
08:52 <ariel@tin> Finished deploy [dumps/dumps@1734c6d]: history dump rebalance script, fixup for extension script dumps, root logger for misc dumps (duration: 00m 02s) [production]
08:52 <ariel@tin> Started deploy [dumps/dumps@1734c6d]: history dump rebalance script, fixup for extension script dumps, root logger for misc dumps [production]
08:40 <gehel> restart relforge1001 to validate latest config changes [production]
08:16 <akosiaris@tin> Finished deploy [citoid/deploy@ba0db9c]: Remove the bad PMCID test from spec (duration: 07m 44s) [production]
08:09 <akosiaris@tin> Started deploy [citoid/deploy@ba0db9c]: Remove the bad PMCID test from spec [production]
08:02 <moritzm> updating HHVM on terbium/wasat to 3.18 [production]
07:57 <akosiaris@tin> Finished deploy [citoid/deploy@ba0db9c]: Remove the bad PMCID test from spec (duration: 00m 38s) [production]
07:56 <akosiaris@tin> Started deploy [citoid/deploy@ba0db9c]: Remove the bad PMCID test from spec [production]
07:48 <akosiaris> schedule 2 hours downtime for all citoid endpoints health on scb boxes [production]
06:08 <marostegui> Deploy alter table s2 - labsdb1003 - T166205 [production]
05:50 <marostegui> Deploy alter table s2 - db1018 - T166205 [production]
05:49 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Add comments to db1018 current status - T166205 (duration: 00m 43s) [production]
05:41 <marostegui> Deploy alter table s4 - dbstore1001 - T166206 [production]
05:22 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1036 - T166205 (duration: 00m 44s) [production]
02:50 <l10nupdate@tin> ResourceLoader cache refresh completed at Thu Jun 15 02:50:16 UTC 2017 (duration 6m 48s) [production]
02:43 <l10nupdate@tin> scap sync-l10n completed (1.30.0-wmf.5) (duration: 07m 34s) [production]
02:26 <l10nupdate@tin> scap sync-l10n completed (1.30.0-wmf.4) (duration: 09m 15s) [production]
01:17 <mutante> releases1001 - reinstalling with stretch [production]
00:15 <mutante> dumpsdata1001 - was reported in icinga as CRIT systemdstate - reason was puppet service was failed with "Invalid value '"no"' for boolean parameter: daemonize" (it was ok on other hosts??). commented the option, stopped puppet, systemctl reset-failed - which made it recover (T165368) [production]
00:02 <twentyafterfour> Deploying phabricator update (tagged release/2017-06-14/1) details: https://phabricator.wikimedia.org/project/view/2831/ [production]
2017-06-14 §
23:55 <mutante> mwreleases: revoke puppet cert, delete salt key, remove from icinga. releases1001 still syncing disks for a while (50m), being created... T164030 [production]
23:49 <mutante> ganeti: removed instance mwreleases1001, created new instance releases1001 with same parameters (2 VCPUS,4G memory, 1 x 128G disk) (T164030) [production]
23:41 <mutante> mwreleases1001 - scheduled downtime, shutdown, kill VM, re-install as releases1001 (T164030) [production]
23:33 <catrope@tin> Synchronized php-1.30.0-wmf.5/includes/: Unbreak watchlist highlighting T167922 (duration: 01m 30s) [production]
23:30 <catrope@tin> Synchronized wmf-config/InitialiseSettings.php: Send search traffic back to eqiad T149006 (duration: 00m 44s) [production]
23:23 <catrope@tin> Synchronized wmf-config/: ORES config cleanups (duration: 00m 46s) [production]
22:43 <reedy@tin> Synchronized php-1.30.0-wmf.5/extensions/WikimediaMaintenance/addWiki.php: Remove accountaudit (duration: 00m 44s) [production]
22:33 <reedy@tin> Synchronized wmf-config/InitialiseSettings.php: meta namespace talk for atjwiki (duration: 00m 44s) [production]
21:36 <reedy@tin> Synchronized wmf-config/interwiki.php: Update interwiki map for atjwiki T167714 (duration: 00m 44s) [production]
21:29 <reedy@tin> Synchronized langlist: Add atj T167714 (duration: 00m 43s) [production]
21:29 <reedy@tin> Synchronized static/images/project-logos/: atjwiki T167714 (duration: 00m 43s) [production]
21:27 <reedy@tin> Synchronized wmf-config/InitialiseSettings.php: atjwiki T167714 (duration: 00m 43s) [production]
21:26 <reedy@tin> rebuilt wikiversions.php and synchronized wikiversions files: Add atjwiki T167714 [production]
21:25 <reedy@tin> Synchronized dblists/: add atjwiki T167714 (duration: 00m 42s) [production]