1-50 of 10000 results (14ms)
2017-10-08 §
21:40 <hoo> Killed "dumpwikidatardf.sh truthy nt" (wikidata truthy dump) on snapshot1007, got stuck after T169680. [production]
11:28 <volans> ack-ed puppet not running on stat100[5-6],snapshot100[1,5-7] due to NFSD stuck on dataset1001 - T169680 [production]
08:19 <elukey> restart varnish backend on cp4026 to stop 503s [production]
2017-10-07 §
22:46 <reedy@tin> Synchronized php-1.31.0-wmf.2/resources/: T177705 (duration: 00m 48s) [production]
22:44 <reedy@tin> Synchronized php-1.31.0-wmf.2/includes/specials/SpecialBlock.php: T177705 (duration: 00m 49s) [production]
2017-10-06 §
18:58 <ejegg> updated CiviCRM from 197b22dce5a6c60a6e3697f5b9ba49de536d0bbb to 276cbfd4ebbf4b1630ab9eb87fe5ea365d624767 [production]
18:38 <catrope@tin> Synchronized php-1.31.0-wmf.2/extensions/TwoColConflict/: Unbreak preference changes (T177524) (duration: 00m 47s) [production]
18:23 <arlolra@tin> Finished deploy [parsoid/deploy@e437c93]: Updating Parsoid to 772e11bf (duration: 12m 41s) [production]
18:10 <arlolra@tin> Started deploy [parsoid/deploy@e437c93]: Updating Parsoid to 772e11bf [production]
15:30 <moritzm> repooling mw2145 (API server). it was previously depooled, but there's no tickets or SAL entries indicating hardware maintenance, so it was probably simply forgotten to repool it [production]
15:25 <moritzm> repooling mw2203 (API server). it was previously depooled, but there's no tickets or SAL entries indicating hardware maintenance, so it was probably simply forgotten to repool it [production]
12:54 <moritzm> repooling mw2248-mw2250 (job runners). they were previosly depooled, but there's no tickets or SAL entries indicating hardware maintenance, so they were probably simply forgotten to repool [production]
11:46 <bblack> cp1* (eqiad caches) - upgrade to nginx-1.13.5-1+wmf1~jessie1 to match all the other sites [production]
11:34 <moritzm> repooling mw2244/mw2245 (image scalers). they were previosly depooled, but there's no tickets or SAL entries indicating hardware maintenance, so they were probably simply forgotten to repool [production]
11:21 <moritzm> installing gdk-pixbuf security updates [production]
11:10 <elukey> rolling restart of all the druid daemons on druid100[1-6] to pick up new logging changes [production]
11:03 <moritzm> installing git security updates on trusty (Debian already fixed) [production]
10:51 <moritzm> upgrading HHVM on script runners to HHVM 3.18.5 [production]
10:47 <moritzm> upgrade remaining video scalers in codfw to HHVM 3.18.5 [production]
10:43 <elukey> restart replication (mysql+ eventlogging_sync) on dbstore1002 after mysql restart [production]
10:17 <moritzm> upgrade remaining job runners in codfw to HHVM 3.18.5 [production]
09:53 <elukey> stop replication (eventlogging_sync + mysql) on dbstore1002 as prep step for mysql restart [production]
09:17 <hashar> Restarted the CI Jenkins for plugin upgrades [production]
09:01 <moritzm> upgrade remaining app servers in codfw to HHVM 3.18.5 [production]
08:40 <moritzm> upgrade remaining image scalers in codfw to HHVM 3.18.5 [production]
08:09 <ema> upgrade pybal to 1.14.0 on eqiad primary LVSs [production]
07:48 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Restore db1076 original weight - T174054 (duration: 00m 48s) [production]
07:23 <ema> upgrade pybal to 1.14.0 on eqiad secondary LVSs [production]
07:18 <moritzm> upgrade remaining API servers in codfw to HHVM 3.18.5 [production]
07:11 <legoktm> live hacking on mwdebug1002 [production]
07:05 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase db1076 weight - T174054 (duration: 00m 47s) [production]
06:37 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase db1076 weight - T174054 (duration: 00m 47s) [production]
06:35 <ema> set max_connections=0 on cp3032 varnish-be [production]
06:23 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Add db1106 to the config - T172679 (duration: 00m 47s) [production]
06:21 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Add db1106 to the config - T172679 (duration: 00m 47s) [production]
06:00 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1076 with low weight - T174054 (duration: 00m 46s) [production]
05:36 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1053 - T174509 (duration: 00m 48s) [production]
05:35 <marostegui> Stop MySQL on db1076 for an upgrade [production]
2017-10-05 §
23:40 <ejegg> updated CiviCRM from cca1a9d90ca41fe9fc75c12095225220dcc43226 to 197b22dce5a6c60a6e3697f5b9ba49de536d0bbb [production]
23:37 <twentyafterfour@tin> Synchronized php-1.31.0-wmf.2/resources/lib/jquery/jquery.migrate.js: sync https://gerrit.wikimedia.org/r/#/c/382530 and https://gerrit.wikimedia.org/r/#/c/382529 (duration: 00m 47s) [production]
23:34 <XioNoX> upgrading ps1-d2-eqiad.mgmt.eqiad.wmnet (unused) [production]
23:23 <twentyafterfour@tin> Synchronized php-1.31.0-wmf.2/extensions/CirrusSearch/includes/BaseInterwikiResolver.php: sync https://gerrit.wikimedia.org/r/#/c/382625/ (duration: 00m 47s) [production]
23:21 <twentyafterfour@tin> Synchronized php-1.31.0-wmf.2/extensions/WikimediaEvents/modules/ext.wikimediaEvents.searchSatisfaction.js: SWAT https://gerrit.wikimedia.org/r/#/c/382611/ (duration: 00m 47s) [production]
22:47 <twentyafterfour@tin> Synchronized php-1.31.0-wmf.2/extensions/CirrusSearch/includes/: Sync CirrusSearch extension again for good measure (duration: 00m 51s) [production]
22:43 <twentyafterfour@tin> Synchronized php-1.31.0-wmf.2/extensions/CirrusSearch: Sync CirrusSearch extension again for good measure (duration: 00m 57s) [production]
22:39 <twentyafterfour@tin> Synchronized wmf-config/CirrusSearch-common.php: sync https://gerrit.wikimedia.org/r/#/c/382474/ (duration: 00m 47s) [production]
22:37 <twentyafterfour@tin> Synchronized php-1.31.0-wmf.2/extensions/CirrusSearch/: deploy https://gerrit.wikimedia.org/r/#/c/382618/ (duration: 00m 58s) [production]
22:13 <awight@tin> Finished deploy [ores/deploy@42c5663]: Cause ORES service restart (duration: 00m 19s) [production]
22:13 <awight@tin> Started deploy [ores/deploy@42c5663]: Cause ORES service restart [production]
22:04 <bd808@tin> Finished deploy [striker/deploy@fbb9019]: Prevent tools from being named with invalid Kubernetes namespace labels (T176681) (duration: 00m 33s) [production]