3301-3350 of 10000 results (63ms)
2018-11-16 §
09:25 <moritzm> installing postgres updates on labsdb1006 [production]
09:21 <moritzm> removed labvirt1016 from debmonitor db, got renamed to cloudvirt1016 [production]
08:40 <moritzm> installing curl security updates on jessie [production]
07:32 <elukey> forced reboot + fsck + removal of /var/lib/hadoop/data/l from fstab on analytics1029 [production]
06:36 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Pool pc2009 in pc3 - T208383 (duration: 00m 56s) [production]
06:28 <marostegui> Set sync_binlog=0 and trx_commit=2 on dbstore2002:3313 to let it catch up [production]
05:47 <vgutierrez> uploaded certcentral 0.7 to apt.wikimedia.org (stretch) - T208967 T209475 [production]
00:55 <mutante> some users reported missing files in home dirs on mwmaint1002, reversed rsyncd/ferm setup and rsynced /home from mwmaint2001 to /root on mwmaint1002, restored individually where requested, rsync is not fully automatic but puppetized with rsync::quickdatacopy [production]
00:35 <sbisson@deploy1001> Synchronized php-1.33.0-wmf.4/extensions/WikimediaEvents/includes/PageViews.php: SWAT: [[gerrit:473861|Exclude users where getRegistration() returns null]] (duration: 00m 47s) [production]
00:26 <eileen> civicrm revision changed from 71755d021b to 6b279509f8, config revision is 684ec9b7c0 (lybunt report) [production]
00:22 <sbisson@deploy1001> Synchronized php-1.33.0-wmf.4/extensions/GrowthExperiments: SWAT: [[gerrit:473843]] [[gerrit:473844]] [[gerrit:473845]] (duration: 00m 49s) [production]
00:21 <sbisson@deploy1001> sync aborted: php-1.33.0-wmf.4/extensions/GrowthExperiments SWAT: [[gerrit:473843]] [[gerrit:473844]] [[gerrit:473845]] (duration: 06m 16s) [production]
00:15 <sbisson@deploy1001> Started scap: php-1.33.0-wmf.4/extensions/GrowthExperiments SWAT: [[gerrit:473843]] [[gerrit:473844]] [[gerrit:473845]] [production]
2018-11-15 §
21:58 <mutante> mwmaint1002 - restoring entire /home of mwmaint1001 from Bacula (job queued and to tmp dir, not directly into /home) [production]
21:06 <hashar> Deleting Nodepool instances on contintcloud T209361 [production]
21:05 <hashar> Stopped nodepool on labnodepool1001.eqiad.wmnet . Service is no more used. T209361 T209642 [production]
20:22 <mholloway-shell@deploy1001> Finished deploy [kartotherian/deploy@UNKNOWN]: Fix: Loosen WDQS content-type header check to unbreak maps (T209471) (duration: 03m 55s) [production]
20:18 <mholloway-shell@deploy1001> Started deploy [kartotherian/deploy@UNKNOWN]: Fix: Loosen WDQS content-type header check to unbreak maps (T209471) [production]
20:17 <mutante> re-added Chase to pwstore, signed .users file, re-encrypted all pwstore files, git pushed [production]
20:15 <mholloway-shell@deploy1001> Finished deploy [kartotherian/deploy@48a1e83]: Fix: Loosen WDQS content-type header check to unbreak maps (T209471) (duration: 04m 26s) [production]
20:11 <mholloway-shell@deploy1001> Started deploy [kartotherian/deploy@48a1e83]: Fix: Loosen WDQS content-type header check to unbreak maps (T209471) [production]
20:04 <urandom> dropping disused keyspaces -- T208616 [production]
20:04 <catrope@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Enable data collection for UnderstandingFirstDay on cswiki and kowiki (duration: 00m 53s) [production]
19:51 <catrope@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Configure sensitive namespaces for EditorJourney schema (T207307) (duration: 00m 53s) [production]
19:49 <catrope@deploy1001> Synchronized php-1.33.0-wmf.4/extensions/WikimediaEvents/: cherry-picks for T208773 (duration: 00m 54s) [production]
18:55 <jforrester@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Hot-deploy I26f2dc2e: Don't over-ride default Wikibase string limits (duration: 00m 53s) [production]
18:50 <jforrester@deploy1001> Synchronized wmf-config/Wikibase.php: Hot-deploy Ib10de2e3: Don't set Wikibase string limits when null (duration: 00m 55s) [production]
18:31 <ladsgroup@deploy1001> Finished deploy [ores/deploy@dba11e9]: Another small update (duration: 13m 42s) [production]
18:18 <ladsgroup@deploy1001> Started deploy [ores/deploy@dba11e9]: Another small update [production]
18:17 <ladsgroup@deploy1001> Finished deploy [ores/deploy@51cdf6b]: T208623 (duration: 14m 41s) [production]
18:03 <ladsgroup@deploy1001> Started deploy [ores/deploy@51cdf6b]: T208623 [production]
17:27 <anomie@mwmaint1002> Running refreshExternallinksIndex.php on section 7 wikis in group 2 for T209373. This may cause lag in codfw. [production]
17:27 <anomie@mwmaint1002> Running refreshExternallinksIndex.php on section 6 wikis in group 2 for T209373. This may cause lag in codfw. [production]
17:27 <anomie@mwmaint1002> Running refreshExternallinksIndex.php on section 5 wikis in group 2 for T209373. This may cause lag in codfw. [production]
17:27 <anomie@mwmaint1002> Running refreshExternallinksIndex.php on section 3 wikis in group 2 for T209373. This may cause lag in codfw. [production]
17:27 <anomie@mwmaint1002> Running refreshExternallinksIndex.php on section 2 wikis in group 2 for T209373. This may cause lag in codfw. [production]
17:27 <anomie@mwmaint1002> Running refreshExternallinksIndex.php on section 1 wikis in group 2 for T209373. This may cause lag in codfw. [production]
17:20 <gehel> upgrade prometheus-blazegraph-exporter on all wdqs nodes - T206123 [production]
16:41 <bstorm_> rebooted labsdb1007 for upgrades [production]
16:37 <andrewbogott> rebuilding labvirt1015 and cloudvirt1015 [production]
14:21 <anomie@mwmaint1002> Running refreshExternallinksIndex.php on wikitech for T209373. This may cause lag in codfw. [production]
14:21 <anomie@mwmaint1002> Running refreshExternallinksIndex.php on section 8 wikis in group 1 for T209373. This may cause lag in codfw. [production]
14:21 <anomie@mwmaint1002> Running refreshExternallinksIndex.php on section 7 wikis in group 1 for T209373. This may cause lag in codfw. [production]
14:21 <anomie@mwmaint1002> Running refreshExternallinksIndex.php on section 5 wikis in group 1 for T209373. This may cause lag in codfw. [production]
14:21 <anomie@mwmaint1002> Running refreshExternallinksIndex.php on section 4 wikis in group 1 for T209373. This may cause lag in codfw. [production]
14:21 <anomie@mwmaint1002> Running refreshExternallinksIndex.php on section 3 wikis in group 1 for T209373. This may cause lag in codfw. [production]
14:20 <anomie@mwmaint1002> Running refreshExternallinksIndex.php on section 2 wikis in group 1 for T209373. This may cause lag in codfw. [production]
14:07 <gehel> plugin and JVM upgrade on elasticsearch / cirrus / eqiad completed - T209293 [production]
13:59 <hashar@deploy1001> rebuilt and synchronized wikiversions files: all wikis to 1.33.0-wmf.4 [production]
13:55 <XioNoX> push firewall policies to pfw3-eqiad - T209421 [production]