3851-3900 of 10000 results (47ms)
2017-06-13 §
15:41 <gehel> restart of relforge1001 to test https://gerrit.wikimedia.org/r/#/c/358353/ [production]
15:09 <gehel> applying new GC configuration on elastic1018 - T167636 [production]
14:53 <godog> update inter-routing for upload to point esams to codfw [production]
14:22 <gehel> restarting elasticsearch on relforge to validate GC configuration - T167636 [production]
14:17 <ottomata> stopping puppet on cp1045, testing removal of xff from varnishkafka webrequest data [production]
14:14 <godog> point upload varnish to swift in codfw - T162609 [production]
14:11 <moritzm> upgrading mw1299-mw1306 to HHVM 3.18 [production]
14:10 <urandom> T164865: Restart RESTBase dev; apply range delete probability of 1.0 [production]
13:30 <godog> Thumbor to group1 wikis + mediawiki.org - T167793 [production]
13:15 <hashar> European SWAT completed [production]
13:13 <hashar@tin> Synchronized php-1.30.0-wmf.4/extensions/Popups: actions/rest: Use DB-key version of title - T167633 (duration: 00m 41s) [production]
13:08 <hashar@tin> Synchronized php-1.30.0-wmf.4/includes/htmlform/OOUIHTMLForm.php: Do not try to parse empty argument in getErrorsOrWarnings in OOUI - T167644 (duration: 00m 41s) [production]
13:04 <hashar@tin> Synchronized wmf-config/Wikibase-production.php: Enable Wikidata echo notifications for all wikis (except enwiki, frwiki, dewiki) - T142102 (duration: 00m 42s) [production]
12:44 <marostegui> Deploy alter table on s2 on db1036 - T166205 [production]
12:39 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1036 - T166205 (duration: 00m 41s) [production]
12:12 <marostegui> Deploy alter table on s2 on dbstore1002 - T166205 [production]
12:11 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1064 - T166206 (duration: 00m 51s) [production]
11:56 <godog> enable thumbor serving for group0 wikis with media files - T167782 [production]
11:41 <moritzm> upgrading HHVM on tin/naos to HHVM 3.18 [production]
10:59 <moritzm> upgrading mw1283-mw1290 to HHVM 3.18 [production]
10:21 <godog> reenable thumbor swift storage, same paths as mediawiki - T167783 [production]
10:11 <elukey> completed rollout of https://gerrit.wikimedia.org/r/354449 [production]
09:54 <moritzm> upgrading mw2248-mw2250 to HHVM 3.18 [production]
09:37 <godog> disable thumbor shadow requests, enable thumbor-only serving for testwiki - T167490 [production]
09:28 <moritzm> upgrading mw1276-mw1282 to HHVM 3.18 [production]
09:27 <elukey> puppet disabled on kafka*, analytics*, druid*, conf* for https://gerrit.wikimedia.org/r/354449 - incremental rollout [production]
09:13 <marostegui> Deploy alter table s4 - db1095 - T166206 [production]
08:56 <moritzm> upgrading mw1165-mw1167 to HHVM 3.18 [production]
08:42 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1060 - T166205 (duration: 00m 41s) [production]
08:38 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Restore db1089 original weight - T166935 (duration: 00m 42s) [production]
08:21 <gehel> restart OSM synchronisation on maps2001 [production]
08:14 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase db1089 weight (duration: 00m 42s) [production]
08:05 <gehel@puppetmaster1001> conftool action : set/pooled=yes; selector: name=elastic2020.codfw.wmnet [production]
08:01 <gehel> adding elastic2020 back in the elasticsearch cluster - T149006 [production]
07:47 <marostegui> Drop table updates on enwiki (s1) - T139342 [production]
07:41 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase db1089 weight (duration: 00m 41s) [production]
07:30 <moritzm> restarting HHVM on mw canaries to pick up libtasn update [production]
07:21 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1089 with less weight (duration: 00m 41s) [production]
07:12 <marostegui> Reboot scb2005 - T167638 [production]
06:55 <elukey> executed "cumin 'mw2*.codfw.wmnet' 'find /var/log/hhvm/* -user root -exec chown www-data:www-data {} \;'" to fix the last occurences of wrong root:adm hhvm log occurrences [production]
06:51 <moritzm> installing libtasn security updates [production]
06:43 <marostegui> Stop MySQL on db1089 to upgrade its raid controller firmware - T166935 [production]
06:43 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1089 - T166935 (duration: 00m 42s) [production]
02:33 <l10nupdate@tin> ResourceLoader cache refresh completed at Tue Jun 13 02:33:23 UTC 2017 (duration 6m 12s) [production]
02:27 <l10nupdate@tin> scap sync-l10n completed (1.30.0-wmf.4) (duration: 08m 00s) [production]
01:29 <papaul> OS install on labtestnet2002 [production]
00:40 <andyrussg@tin> Finished scap: Update CentralNotice (duration: 20m 51s) [production]
00:19 <andyrussg@tin> Started scap: Update CentralNotice [production]
2017-06-12 §
23:22 <mutante> netmon1002 - keyholder arm - loaded rancid deploy key (uses separate passphrase from deployment key) [production]
22:01 <mutante> netmon1002 - apt-get -t jessie-backports install rancid (upgrade from 2.3.8 to 3.6.2 to match version on netmon1001) - rancid version is not specified in puppet so even though backports gets enabled the older version gets installed and this manual step is needed unless we start specifying the version in the manifest (T159756) [production]