|
2018-11-16
§
|
| 14:28 |
<oblivian@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=mw126[1-6].*,dc=eqiad,cluster=appserver |
[production] |
| 14:26 |
<_joe_> |
repooling the mw canaries |
[production] |
| 13:02 |
<moritzm> |
installing spamassassin security update on mendelevium |
[production] |
| 12:10 |
<godog> |
reboot restbase1014, nothing on console |
[production] |
| 11:13 |
<kartik@deploy1001> |
Finished deploy [cxserver/deploy@473b0de]: Update cxserver to b7cdb26 (T208831, T203077, T203160, T206777) (duration: 04m 26s) |
[production] |
| 11:08 |
<kartik@deploy1001> |
Started deploy [cxserver/deploy@473b0de]: Update cxserver to b7cdb26 (T208831, T203077, T203160, T206777) |
[production] |
| 10:39 |
<akosiaris> |
upgrade OTRS to 5.0.32 T209691 |
[production] |
| 09:31 |
<marostegui> |
Set back sync_binlog=1 and trx_commit=1 after dbstore2002:3313 has caught up |
[production] |
| 09:25 |
<moritzm> |
installing postgres updates on labsdb1006 |
[production] |
| 09:21 |
<moritzm> |
removed labvirt1016 from debmonitor db, got renamed to cloudvirt1016 |
[production] |
| 08:40 |
<moritzm> |
installing curl security updates on jessie |
[production] |
| 07:32 |
<elukey> |
forced reboot + fsck + removal of /var/lib/hadoop/data/l from fstab on analytics1029 |
[production] |
| 06:36 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Pool pc2009 in pc3 - T208383 (duration: 00m 56s) |
[production] |
| 06:28 |
<marostegui> |
Set sync_binlog=0 and trx_commit=2 on dbstore2002:3313 to let it catch up |
[production] |
| 05:47 |
<vgutierrez> |
uploaded certcentral 0.7 to apt.wikimedia.org (stretch) - T208967 T209475 |
[production] |
| 00:55 |
<mutante> |
some users reported missing files in home dirs on mwmaint1002, reversed rsyncd/ferm setup and rsynced /home from mwmaint2001 to /root on mwmaint1002, restored individually where requested, rsync is not fully automatic but puppetized with rsync::quickdatacopy |
[production] |
| 00:35 |
<sbisson@deploy1001> |
Synchronized php-1.33.0-wmf.4/extensions/WikimediaEvents/includes/PageViews.php: SWAT: [[gerrit:473861|Exclude users where getRegistration() returns null]] (duration: 00m 47s) |
[production] |
| 00:26 |
<eileen> |
civicrm revision changed from 71755d021b to 6b279509f8, config revision is 684ec9b7c0 (lybunt report) |
[production] |
| 00:22 |
<sbisson@deploy1001> |
Synchronized php-1.33.0-wmf.4/extensions/GrowthExperiments: SWAT: [[gerrit:473843]] [[gerrit:473844]] [[gerrit:473845]] (duration: 00m 49s) |
[production] |
| 00:21 |
<sbisson@deploy1001> |
sync aborted: php-1.33.0-wmf.4/extensions/GrowthExperiments SWAT: [[gerrit:473843]] [[gerrit:473844]] [[gerrit:473845]] (duration: 06m 16s) |
[production] |
| 00:15 |
<sbisson@deploy1001> |
Started scap: php-1.33.0-wmf.4/extensions/GrowthExperiments SWAT: [[gerrit:473843]] [[gerrit:473844]] [[gerrit:473845]] |
[production] |
|
2018-11-15
§
|
| 21:58 |
<mutante> |
mwmaint1002 - restoring entire /home of mwmaint1001 from Bacula (job queued and to tmp dir, not directly into /home) |
[production] |
| 21:06 |
<hashar> |
Deleting Nodepool instances on contintcloud T209361 |
[production] |
| 21:05 |
<hashar> |
Stopped nodepool on labnodepool1001.eqiad.wmnet . Service is no more used. T209361 T209642 |
[production] |
| 20:22 |
<mholloway-shell@deploy1001> |
Finished deploy [kartotherian/deploy@UNKNOWN]: Fix: Loosen WDQS content-type header check to unbreak maps (T209471) (duration: 03m 55s) |
[production] |
| 20:18 |
<mholloway-shell@deploy1001> |
Started deploy [kartotherian/deploy@UNKNOWN]: Fix: Loosen WDQS content-type header check to unbreak maps (T209471) |
[production] |
| 20:17 |
<mutante> |
re-added Chase to pwstore, signed .users file, re-encrypted all pwstore files, git pushed |
[production] |
| 20:15 |
<mholloway-shell@deploy1001> |
Finished deploy [kartotherian/deploy@48a1e83]: Fix: Loosen WDQS content-type header check to unbreak maps (T209471) (duration: 04m 26s) |
[production] |
| 20:11 |
<mholloway-shell@deploy1001> |
Started deploy [kartotherian/deploy@48a1e83]: Fix: Loosen WDQS content-type header check to unbreak maps (T209471) |
[production] |
| 20:04 |
<urandom> |
dropping disused keyspaces -- T208616 |
[production] |
| 20:04 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable data collection for UnderstandingFirstDay on cswiki and kowiki (duration: 00m 53s) |
[production] |
| 19:51 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Configure sensitive namespaces for EditorJourney schema (T207307) (duration: 00m 53s) |
[production] |
| 19:49 |
<catrope@deploy1001> |
Synchronized php-1.33.0-wmf.4/extensions/WikimediaEvents/: cherry-picks for T208773 (duration: 00m 54s) |
[production] |
| 18:55 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Hot-deploy I26f2dc2e: Don't over-ride default Wikibase string limits (duration: 00m 53s) |
[production] |
| 18:50 |
<jforrester@deploy1001> |
Synchronized wmf-config/Wikibase.php: Hot-deploy Ib10de2e3: Don't set Wikibase string limits when null (duration: 00m 55s) |
[production] |
| 18:31 |
<ladsgroup@deploy1001> |
Finished deploy [ores/deploy@dba11e9]: Another small update (duration: 13m 42s) |
[production] |
| 18:18 |
<ladsgroup@deploy1001> |
Started deploy [ores/deploy@dba11e9]: Another small update |
[production] |
| 18:17 |
<ladsgroup@deploy1001> |
Finished deploy [ores/deploy@51cdf6b]: T208623 (duration: 14m 41s) |
[production] |
| 18:03 |
<ladsgroup@deploy1001> |
Started deploy [ores/deploy@51cdf6b]: T208623 |
[production] |
| 17:27 |
<anomie@mwmaint1002> |
Running refreshExternallinksIndex.php on section 7 wikis in group 2 for T209373. This may cause lag in codfw. |
[production] |
| 17:27 |
<anomie@mwmaint1002> |
Running refreshExternallinksIndex.php on section 6 wikis in group 2 for T209373. This may cause lag in codfw. |
[production] |
| 17:27 |
<anomie@mwmaint1002> |
Running refreshExternallinksIndex.php on section 5 wikis in group 2 for T209373. This may cause lag in codfw. |
[production] |
| 17:27 |
<anomie@mwmaint1002> |
Running refreshExternallinksIndex.php on section 3 wikis in group 2 for T209373. This may cause lag in codfw. |
[production] |
| 17:27 |
<anomie@mwmaint1002> |
Running refreshExternallinksIndex.php on section 2 wikis in group 2 for T209373. This may cause lag in codfw. |
[production] |
| 17:27 |
<anomie@mwmaint1002> |
Running refreshExternallinksIndex.php on section 1 wikis in group 2 for T209373. This may cause lag in codfw. |
[production] |
| 17:20 |
<gehel> |
upgrade prometheus-blazegraph-exporter on all wdqs nodes - T206123 |
[production] |
| 16:41 |
<bstorm_> |
rebooted labsdb1007 for upgrades |
[production] |
| 16:37 |
<andrewbogott> |
rebuilding labvirt1015 and cloudvirt1015 |
[production] |
| 14:21 |
<anomie@mwmaint1002> |
Running refreshExternallinksIndex.php on wikitech for T209373. This may cause lag in codfw. |
[production] |
| 14:21 |
<anomie@mwmaint1002> |
Running refreshExternallinksIndex.php on section 8 wikis in group 1 for T209373. This may cause lag in codfw. |
[production] |