751-800 of 10000 results (49ms)
2017-09-01 §
09:10 <godog> bounce graphite-web on graphite1001, problematic query? using all CPU [production]
09:04 <ema> lvs1007 upgrade to pybal 1.13.11 - one-packet-scheduling, instrumentation fixes. T104442, T103882 [production]
08:59 <godog> depool restbase200[135] before reimage - T169939 [production]
07:22 <elukey> restart apache2 and hue on thorium, Analytics sites down, investigating [production]
07:06 <marostegui> Power reset db2044 as it is unresponsive - T174764 [production]
03:54 <krinkle@tin> Synchronized wmf-config/InitialiseSettings.php: I4dfc33f66c3 - Enable jQuery 3 on nlwiki sister projects (duration: 00m 43s) [production]
00:57 <tstarling@tin> Synchronized php-1.30.0-wmf.16/includes/parser/Parser.php: (no justification provided) (duration: 00m 44s) [production]
00:13 <RainbowSprinkles> gerrit: flushed all non-login caches, things might be sluggish for the next ~15mins or so [production]
2017-08-31 §
20:03 <demon@tin> rebuilt wikiversions.php and synchronized wikiversions files: (no justification provided) [production]
19:27 <mattflaschen@tin> Finished scap: Watchlist filters: Convert edit watchlist button to new UX and fix server-side tag filtering. T172030 (duration: 21m 23s) [production]
19:17 <cmjohnson1> powercycled mw1294 [production]
19:05 <mattflaschen@tin> Started scap: Watchlist filters: Convert edit watchlist button to new UX and fix server-side tag filtering. T172030 [production]
18:53 <mattflaschen@tin> Synchronized docroot/noc/conf/interwiki-labs.php.txt: Interwiki links on Beta Cluster: T69931 (duration: 00m 47s) [production]
18:51 <mattflaschen@tin> Synchronized wmf-config: Interwiki links on Beta Cluster: T69931 (duration: 00m 49s) [production]
17:50 <demon@tin> Synchronized php-1.30.0-wmf.16/skins/MinervaNeue/resources/skins.minerva.icons.images.scripts/watch.svg: sorta impromptu swat thing (duration: 00m 47s) [production]
16:35 <ejegg> re-enabled donation queue consumer and thank you mail sender [production]
15:25 <ejegg> updated civicrm from f00611d9aa3ae4a1ffbff70dcca90c7ae07b6c68 to 820bd98c7c23a851c38a75111c8c1532c781e198 [production]
15:20 <ejegg> updated SmashPig from 8f4388d407f313c1083cf96ed106efcc5d5a29f7 to 8eb98c10ff1129ea65dd81559656001c819cc12f [production]
15:19 <ejegg> turned off donation queue consumer and thank you job [production]
15:18 <elukey> restart zookeeper on conf200[2,3] for jvm security updates [production]
13:36 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Restore db1084 original weight - T168661 (duration: 00m 47s) [production]
13:10 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase db1084 weight - T168661 (duration: 00m 47s) [production]
13:07 <elukey> restart zookeeper on conf2001 for security updates (canary node) [production]
13:00 <moritzm> installing augeas security updates [production]
12:34 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1084 with low weight - T168661 (duration: 00m 47s) [production]
12:14 <marostegui> Upgrade MariaDB on db1084 - T168661 [production]
12:11 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1084 - T168661 (duration: 00m 47s) [production]
12:05 <akosiaris> reimage chlorine and kubernetes1004 T170119 [production]
12:04 <akosiaris> reimage chlorine and kubernetes1004 T10119 [production]
11:58 <moritzm> restarting nginx on dataset1001/ms1001 to pick up libxml security update [production]
11:21 <elukey> restart apache2 on bohrium for libxml + gnutls security updates [production]
11:18 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Restore db1056 original weight - T168661 (duration: 00m 47s) [production]
11:17 <moritzm> restarting apache on dbmonitor* to pick up libxml security update [production]
10:27 <moritzm> restarting nginx on prometheus* to pick up libxml security update [production]
10:22 <moritzm> restarting apache on einsteinium/tegmen (Icinga hosts) to pick up libxml security update [production]
10:14 <moritzm> restarting nginx on meitnerium/archiva.wikimedia.org to pick up libxml security update [production]
10:13 <moritzm> restarting nginx on debug proxies to pick up libxml security update [production]
10:10 <moritzm> restarting apache on hafnium to pick up libxml security update [production]
09:57 <addshore> extensions/Cognate/maintenance/recalculateCognateNormalizedHashes.php run done, 6326 hashes recalculated, T172987 [production]
09:50 <addshore> addshore@terbium:~$ mwscript extensions/Cognate/maintenance/recalculateCognateNormalizedHashes.php --wiki enwiktionary --batch-size 1000000 # Should upsert 6326 rows T172987 [production]
09:44 <addshore> mwscript extensions/Cognate/maintenance/recalculateCognateNormalizedHashes.php --wiki enwiktionary --batch-size 1000000 --dry-run [production]
09:43 <moritzm> restarting apache on auth* to pick up libxml security update [production]
09:40 <moritzm> installing libxml2 security updates [production]
09:10 <moritzm> installing experimental hhvm-luasandbox package on mw1261 (canary app server) with backported patch for T171392 / T173705 [production]
09:04 <hashar> contint1001 upgrading git, restarting git-daemon and zuul-merger - T161086 [production]
09:01 <godog> test reimage of restbase2001 - T169939 [production]
08:57 <hashar> contint2001 upgrading git, restarting git-daemon and zuul-merger - T161086 [production]
08:45 <godog> remove /srv/deployment/grafana on naos and tin, unused - T170881 [production]
08:41 <gehel> restart postgresql / kartotherian / tilerator / tileratorui on maps@eqiad [production]
08:40 <addshore@tin> Synchronized php-1.30.0-wmf.16/extensions/Cognate/maintenance/recalculateCognateNormalizedHashes.php: T172987 [[gerrit:374957|Add waitForReplication to RecalculateCognateNormalizedHashes script]] (duration: 00m 47s) [production]