|
2019-02-06
ยง
|
| 14:29 |
<elukey> |
add term mysql-dbstore to analytics-in4/6 on cr1/2-eqiad to allow tcp connections to dbstore100[3-5] - T210478 |
[production] |
| 12:30 |
<vgutierrez@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=cp3032.esams.wmnet |
[production] |
| 12:29 |
<vgutierrez@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=cp3032.esams.wmnet |
[production] |
| 12:28 |
<vgutierrez@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=cp3033.esams.wmnet |
[production] |
| 12:26 |
<vgutierrez@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=cp3033.esams.wmnet |
[production] |
| 12:25 |
<vgutierrez@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=cp3040.esams.wmnet |
[production] |
| 12:24 |
<vgutierrez@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=cp3040.esams.wmnet |
[production] |
| 12:22 |
<vgutierrez@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=cp3041.esams.wmnet |
[production] |
| 12:22 |
<Amir1> |
EU SWAT is done |
[production] |
| 12:21 |
<vgutierrez@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=cp3041.esams.wmnet |
[production] |
| 12:20 |
<vgutierrez> |
restarting varnish-fe safely across esams/text cluster - T215389 |
[production] |
| 12:19 |
<ladsgroup@deploy1001> |
Synchronized wmf-config/Wikibase.php: SWAT: [[gerrit:488114|Use separate DB connection for ID insertions on testwikidatawiki (T215147)]], Part II (duration: 00m 54s) |
[production] |
| 12:17 |
<ladsgroup@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:488114|Use separate DB connection for ID insertions on testwikidatawiki (T215147)]], Part I (duration: 00m 55s) |
[production] |
| 11:58 |
<vgutierrez@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=cp3042.esams.wmnet |
[production] |
| 11:57 |
<vgutierrez@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=cp3042.esams.wmnet |
[production] |
| 11:56 |
<vgutierrez> |
restarting varnish-fe in cp3042 - T215389 |
[production] |
| 11:02 |
<_joe_> |
restarting nginx safely across the appserver fleets in order to be able to run puppet without errors |
[production] |
| 10:41 |
<marostegui> |
Revoke access to testreduce from ruthenium on m5 - https://phabricator.wikimedia.org/T214740 |
[production] |
| 10:04 |
<moritzm> |
reimaging graphite2002 to buster |
[production] |
| 10:01 |
<akosiaris> |
restart varnish-frontend on cp3030 T215389 |
[production] |
| 10:01 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1091 (duration: 00m 52s) |
[production] |
| 09:33 |
<marostegui> |
Remove wikiuser from dbstore1003-dbstore1005 T210478 |
[production] |
| 09:15 |
<godog> |
swift codfw-prod: more weight for ms-be2047 - T209395 T209921 |
[production] |
| 09:00 |
<marostegui> |
Create research_role on dbstore1003-1005 on all instances - T214469 |
[production] |
| 08:48 |
<marostegui> |
Deploy schema change on db1091 - T210713 |
[production] |
| 08:48 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1091 (duration: 00m 53s) |
[production] |
| 08:24 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1121 (duration: 00m 53s) |
[production] |
| 07:51 |
<marostegui> |
Deploy schema change on db1121 - this will generate lag on s4 labs - also upgrade MySQL on db1121 T210713 |
[production] |
| 07:51 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1121 (duration: 00m 54s) |
[production] |
| 07:47 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1103:3314 (duration: 00m 54s) |
[production] |
| 07:19 |
<marostegui> |
Deploy schema change on wikitech T210713 |
[production] |
| 07:14 |
<marostegui> |
Stop 's4' slave on dbstore1002 |
[production] |
| 07:13 |
<marostegui> |
Deploy schema change on db1103:3314 (db1097:3314 was also done previously) - T210713 |
[production] |
| 07:13 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1103:3314 (duration: 00m 53s) |
[production] |
| 07:09 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1097:3314 (duration: 00m 56s) |
[production] |
| 06:29 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1097:3314 (duration: 01m 06s) |
[production] |
| 04:39 |
<mutante> |
reloaded icinga service, cant find new check command definition |
[production] |
| 03:13 |
<twentyafterfour@deploy1001> |
Finished scap: testwikis wikis to 1.33.0-wmf.16 refs T206670 (duration: 04m 18s) |
[production] |
| 03:09 |
<twentyafterfour@deploy1001> |
Started scap: testwikis wikis to 1.33.0-wmf.16 refs T206670 |
[production] |
| 03:05 |
<mutante> |
actinium - gzipping and rotating some access logs |
[production] |
| 03:01 |
<twentyafterfour@deploy1001> |
Synchronized scap/plugins/updateinterwikicache.py: (no justification provided) (duration: 00m 55s) |
[production] |
| 02:47 |
<mutante> |
actinium - blocking a bad domain and restarting squid3 |
[production] |
| 02:40 |
<twentyafterfour@deploy1001> |
Finished scap: sync and update localization for 1.33.0-wmf.16 (duration: 15m 50s) |
[production] |
| 02:32 |
<XioNoX> |
push firewall rule to pfw3-eqiad - T215364 |
[production] |
| 02:27 |
<mutante> |
actinium - apt-get clean for 8% more disk space after icinga alert |
[production] |
| 02:25 |
<twentyafterfour@deploy1001> |
Started scap: sync and update localization for 1.33.0-wmf.16 |
[production] |
| 02:16 |
<twentyafterfour@deploy1001> |
rebuilt and synchronized wikiversions files: all wikis to 1.33.0-wmf.14 refs T206670 |
[production] |
| 02:12 |
<eileen> |
civicrm revision changed from 6042acb363 to 613b388916, config revision is 306b4de48f |
[production] |
| 02:02 |
<twentyafterfour@deploy1001> |
scap failed: average error rate on 7/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/db09a36be5ed3e81155041f7d46ad040 for details) |
[production] |
| 01:22 |
<XioNoX> |
remove peering4/6 prefix-list from routers |
[production] |