|
2020-03-04
§
|
| 22:09 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 22:02 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 22:00 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 21:46 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 21:46 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 21:46 |
<urbanecm@deploy1001> |
Synchronized wmf-config/interwiki.php: Update interwiki cache (duration: 02m 25s) |
[production] |
| 21:43 |
<urbanecm@deploy1001> |
Synchronized dblists/special.dblist: 8decd01: Add gewikimedia to special wikis (duration: 01m 06s) |
[production] |
| 21:35 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 21:28 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 21:16 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'canary' . |
[production] |
| 21:16 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 20:54 |
<eileen> |
process-control config revision is 21eb2e891f |
[production] |
| 20:29 |
<otto@deploy1001> |
Synchronized php-1.35.0-wmf.22/extensions/WikimediaEvents/modules/ext.wikimediaEvents/clientError.js: [[gerrit:576942|Include required url in mediawiki/client/error event (T246030)]] (duration: 01m 05s) |
[production] |
| 19:58 |
<otto@deploy1001> |
Synchronized php-1.35.0-wmf.22/extensions/WikimediaEvents/modules/ext.wikimediaEvents/clientError.js: SWAT: [[gerrit:576931|Fix callback parameters for client error logging (T246030)]] (duration: 01m 07s) |
[production] |
| 19:52 |
<shdubsh> |
restart logstash on logstash2005 -- testing field type mismatch mitigation |
[production] |
| 18:43 |
<mutante> |
starting new DHCP servers to confirm they work and letting puppet immediately stop them again to clear systemd status |
[production] |
| 18:30 |
<mutante> |
notebook1003 - restarted nagios-nrpe-server |
[production] |
| 17:41 |
<addshore> |
stop item term rebuild at Q Q60345318 as I generate more lists (T219123) |
[production] |
| 16:50 |
<otto@cumin1001> |
conftool action : set/pooled=true; selector: dnsdisc=eventgate-analytics-external |
[production] |
| 16:49 |
<cmjohnson@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 16:45 |
<cmjohnson@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 16:05 |
<ottomata> |
destroying unused eventgate-main 'main' and eventgate-analytics 'analytics' helm releases - installed: false |
[production] |
| 16:02 |
<addshore> |
addshore@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/rebuildPropertyTerms.php --wiki=wikidatawiki --sleep 1 --batch-size=50 # T244115 |
[production] |
| 15:47 |
<liw@deploy1001> |
Synchronized php: group1 wikis to 1.35.0-wmf.22 (duration: 01m 03s) |
[production] |
| 15:46 |
<liw@deploy1001> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.35.0-wmf.22 |
[production] |
| 15:29 |
<vgutierrez> |
upgrading ATS to version 8.0.6 on eqiad |
[production] |
| 15:11 |
<thcipriani> |
restarting zuul |
[production] |
| 14:55 |
<akosiaris@cumin1001> |
conftool action : set/pooled=true; selector: dnsdisc=eventgate-analytics-external |
[production] |
| 14:44 |
<filippo@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 14:41 |
<filippo@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 14:25 |
<vgutierrez> |
upgrading ATS to version 8.0.6 on codfw |
[production] |
| 14:19 |
<liw@deploy1001> |
rebuilt and synchronized wikiversions files: Revert "group1 wikis to 1.35.0-wmf.21 |
[production] |
| 14:18 |
<akosiaris> |
cleanup old LVS eventgate services. T245203 |
[production] |
| 14:13 |
<addshore> |
cache warming stopped on db1126 and db1111 |
[production] |
| 14:08 |
<liw@deploy1001> |
Synchronized php: group1 wikis to 1.35.0-wmf.22 (duration: 01m 04s) |
[production] |
| 14:07 |
<liw@deploy1001> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.35.0-wmf.22 |
[production] |
| 13:47 |
<addshore> |
START warm cache for db1111 & db1126 for Q25-30 million T219123 (pass 3) |
[production] |
| 13:33 |
<godog> |
disable puppet on install1002 to test partman on theemin |
[production] |
| 13:19 |
<vgutierrez> |
upgrading ATS to version 8.0.6 on esams |
[production] |
| 13:14 |
<marostegui> |
Drop fixcopyrightwiki from sanitarium hosts (db1112, db2074) to avoid getting the data alert - T246055 |
[production] |
| 12:55 |
<urbanecm@deploy1001> |
Synchronized wmf-config/throttle.php: 37db2a1: Add new throttle rule for WikiGap Göteborg 2020-03-06 (T246888) (duration: 01m 04s) |
[production] |
| 12:23 |
<XioNoX> |
add flowspec rule on cr3-knams - T243482 |
[production] |
| 12:20 |
<Urbanecm> |
EU SWAT done |
[production] |
| 12:19 |
<moritzm> |
installing 4.9.210-1~deb8u1 kernel on jessie hosts (no reboots, just the upgrade) |
[production] |
| 12:19 |
<urbanecm@deploy1001> |
Synchronized php-1.35.0-wmf.22/extensions/GrowthExperiments/includes/HelpPanel/QuestionStore.php: SWAT: d495f4c: Replace loadRevisionFromId which has been removed in I0c8fe834da79c (duration: 01m 06s) |
[production] |
| 12:14 |
<urbanecm@deploy1001> |
Synchronized wmf-config/throttle.php: SWAT: 1fa9dda: IP Cap Lift for University of Mannheim Wikimedia Event (2020-04-01) (T246832) (duration: 01m 06s) |
[production] |
| 12:11 |
<moritzm> |
imported linux-meta 1.23 to apt.wikimedia.org for jessie-wikimedia |
[production] |
| 12:04 |
<urbanecm@deploy1001> |
Synchronized wmf-config/throttle.php: SWAT: 85a5c05: Add throttle exempt for 2020-03-07 GenderGap Event (T246813) (duration: 01m 05s) |
[production] |
| 11:51 |
<addshore> |
START warm cache for db1111 & db1126 for Q25-30 million T219123 (pass 2) |
[production] |
| 11:19 |
<vgutierrez> |
upgrading ATS to version 8.0.6 on eqsin |
[production] |