3251-3300 of 10000 results (84ms)
2018-11-19 §
09:10 <Nikerabbit> Rebuilt message group stats cache for T208521 [production]
08:43 <banyek> executing schema change on db2095 (T85757) [production]
07:57 <marostegui> Stop MySQL on db1123 - T209754 [production]
07:56 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1123 to clone db1078 T209754 (duration: 00m 47s) [production]
06:21 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Add db1078 line back to config file but depooled T209754 (duration: 00m 51s) [production]
06:20 <marostegui@deploy1001> sync-file aborted: Add db1078 line back to config file but depooled T209754 (duration: 00m 02s) [production]
2018-11-18 §
17:26 <andrewbogott> restarting cp1078 from mgmt console [production]
09:00 <elukey> cleaned up analytics1039 and restarted Yarn [production]
03:46 <Zoranzoki21> Starting equalization translations for Serbian Cyrillic (sr-ec) and Serbian Latin (sr-el). Possible lags on translatewiki.net due to excessive number of operations [production]
2018-11-17 §
07:02 <volans|off> 'reset modified attributes' on IcingaUI for db1078 (and mgmt) and all its services [production]
06:38 <oblivian@deploy1001> Synchronized wmf-config/db-eqiad.php: Depooling db1078 (duration: 00m 59s) [production]
02:54 <RoanKattouw> Deployed patches for T208112, T208109, T208110 [production]
2018-11-16 §
23:13 <smalyshev@deploy1001> Finished deploy [wdqs/wdqs@ee91c41]: Deploy on test wdqs1010 (duration: 00m 23s) [production]
23:12 <smalyshev@deploy1001> Started deploy [wdqs/wdqs@ee91c41]: Deploy on test wdqs1010 [production]
16:23 <Trey314159> reindexing Chinese wikis on elastic@eqiad and elastic@codfw (T209156) [production]
15:46 <moritzm> rebooting debmonitor* instances for kernel security update and to pick up SSBD [production]
15:01 <hashar> restarting zuul with 1300 events to process [production]
14:56 <marostegui> Create ipblocks_restrictions on labswiki and labtestwiki on db1073 - T209674 [production]
14:56 <ema> upgrade cp-ats to 8.0.0-1wm2 T204225 T204209 [production]
14:39 <ema> trafficserver 8.0.0-1wm2 uploaded to stretch-wikimedia T204225 T204209 [production]
14:36 <hashar> Gracefully stopping zuul (kill -SIGUSR1) [production]
14:29 <_joe_> re-depooling mw1261 for php-fpm testing [production]
14:28 <oblivian@puppetmaster1001> conftool action : set/pooled=yes; selector: name=mw126[1-6].*,dc=eqiad,cluster=appserver [production]
14:26 <_joe_> repooling the mw canaries [production]
13:02 <moritzm> installing spamassassin security update on mendelevium [production]
12:10 <godog> reboot restbase1014, nothing on console [production]
11:13 <kartik@deploy1001> Finished deploy [cxserver/deploy@473b0de]: Update cxserver to b7cdb26 (T208831, T203077, T203160, T206777) (duration: 04m 26s) [production]
11:08 <kartik@deploy1001> Started deploy [cxserver/deploy@473b0de]: Update cxserver to b7cdb26 (T208831, T203077, T203160, T206777) [production]
10:39 <akosiaris> upgrade OTRS to 5.0.32 T209691 [production]
09:31 <marostegui> Set back sync_binlog=1 and trx_commit=1 after dbstore2002:3313 has caught up [production]
09:25 <moritzm> installing postgres updates on labsdb1006 [production]
09:21 <moritzm> removed labvirt1016 from debmonitor db, got renamed to cloudvirt1016 [production]
08:40 <moritzm> installing curl security updates on jessie [production]
07:32 <elukey> forced reboot + fsck + removal of /var/lib/hadoop/data/l from fstab on analytics1029 [production]
06:36 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Pool pc2009 in pc3 - T208383 (duration: 00m 56s) [production]
06:28 <marostegui> Set sync_binlog=0 and trx_commit=2 on dbstore2002:3313 to let it catch up [production]
05:47 <vgutierrez> uploaded certcentral 0.7 to apt.wikimedia.org (stretch) - T208967 T209475 [production]
00:55 <mutante> some users reported missing files in home dirs on mwmaint1002, reversed rsyncd/ferm setup and rsynced /home from mwmaint2001 to /root on mwmaint1002, restored individually where requested, rsync is not fully automatic but puppetized with rsync::quickdatacopy [production]
00:35 <sbisson@deploy1001> Synchronized php-1.33.0-wmf.4/extensions/WikimediaEvents/includes/PageViews.php: SWAT: [[gerrit:473861|Exclude users where getRegistration() returns null]] (duration: 00m 47s) [production]
00:26 <eileen> civicrm revision changed from 71755d021b to 6b279509f8, config revision is 684ec9b7c0 (lybunt report) [production]
00:22 <sbisson@deploy1001> Synchronized php-1.33.0-wmf.4/extensions/GrowthExperiments: SWAT: [[gerrit:473843]] [[gerrit:473844]] [[gerrit:473845]] (duration: 00m 49s) [production]
00:21 <sbisson@deploy1001> sync aborted: php-1.33.0-wmf.4/extensions/GrowthExperiments SWAT: [[gerrit:473843]] [[gerrit:473844]] [[gerrit:473845]] (duration: 06m 16s) [production]
00:15 <sbisson@deploy1001> Started scap: php-1.33.0-wmf.4/extensions/GrowthExperiments SWAT: [[gerrit:473843]] [[gerrit:473844]] [[gerrit:473845]] [production]
2018-11-15 §
21:58 <mutante> mwmaint1002 - restoring entire /home of mwmaint1001 from Bacula (job queued and to tmp dir, not directly into /home) [production]
21:06 <hashar> Deleting Nodepool instances on contintcloud T209361 [production]
21:05 <hashar> Stopped nodepool on labnodepool1001.eqiad.wmnet . Service is no more used. T209361 T209642 [production]
20:22 <mholloway-shell@deploy1001> Finished deploy [kartotherian/deploy@UNKNOWN]: Fix: Loosen WDQS content-type header check to unbreak maps (T209471) (duration: 03m 55s) [production]
20:18 <mholloway-shell@deploy1001> Started deploy [kartotherian/deploy@UNKNOWN]: Fix: Loosen WDQS content-type header check to unbreak maps (T209471) [production]
20:17 <mutante> re-added Chase to pwstore, signed .users file, re-encrypted all pwstore files, git pushed [production]
20:15 <mholloway-shell@deploy1001> Finished deploy [kartotherian/deploy@48a1e83]: Fix: Loosen WDQS content-type header check to unbreak maps (T209471) (duration: 04m 26s) [production]