|
2020-03-04
ยง
|
| 22:51 |
<dzahn@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 22:51 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 22:33 |
<Krenair> |
Shutoff cloudinfra-internal-puppetmaster01, replaced with -02 per T241719 |
[cloudinfra] |
| 22:22 |
<andrewbogott> |
upgrading designate on cloudservices1003/1004 to Queens |
[admin] |
| 22:17 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 22:15 |
<James_F> |
Zuul: Archive CI for LiveTranslate T196452 |
[releng] |
| 22:13 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 22:12 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 22:12 |
<James_F> |
Zuul: Add CI for mediawiki/extensions/PushNotifications |
[releng] |
| 22:10 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 22:10 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 22:09 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 22:09 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 22:09 |
<andrewbogott> |
moving cloudvirt1006 into the maintenance aggregate for T246908 |
[admin] |
| 22:07 |
<RhinosF1> |
deploying hacked test in ##rhinosf1 |
[tools.zppixbot] |
| 22:02 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 22:00 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 21:54 |
<RhinosF1> |
- placing code for T246922 in sftp://login.tools.wmflabs.org/mnt/nfs/labstore-secondary-tools-project/zppixbot |
[tools.zppixbot] |
| 21:46 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 21:46 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 21:46 |
<urbanecm@deploy1001> |
Synchronized wmf-config/interwiki.php: Update interwiki cache (duration: 02m 25s) |
[production] |
| 21:43 |
<urbanecm@deploy1001> |
Synchronized dblists/special.dblist: 8decd01: Add gewikimedia to special wikis (duration: 01m 06s) |
[production] |
| 21:37 |
<bd808> |
Running wmcs-wikireplica-dns to add service names for ngwikimedia.*.db.svc.eqiad.wmflabs (T240772) |
[admin] |
| 21:35 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 21:28 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 21:16 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'canary' . |
[production] |
| 21:16 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 21:14 |
<bd808> |
Running `sudo maintain-meta_p --all-databases --purge` on labsdb1009 (T246056) |
[admin] |
| 21:10 |
<bd808> |
Running `sudo maintain-meta_p --all-databases --purge` on labsdb1010 (T246056) |
[admin] |
| 21:08 |
<bd808> |
Running `sudo maintain-meta_p --all-databases --purge` on labsdb1011 (T246056) |
[admin] |
| 21:05 |
<bd808> |
Running `sudo maintain-meta_p --all-databases --purge` on labsdb1002 (T246056) |
[admin] |
| 20:59 |
<hashar> |
deployment-puppetmaster04: /var/log/syslog /var/log/debug and /var/log/daemon.log are 5GB filling / |
[releng] |
| 20:59 |
<James_F> |
Deleting deployment-parsoid09 for T246395 |
[releng] |
| 20:54 |
<eileen> |
process-control config revision is 21eb2e891f |
[production] |
| 20:29 |
<otto@deploy1001> |
Synchronized php-1.35.0-wmf.22/extensions/WikimediaEvents/modules/ext.wikimediaEvents/clientError.js: [[gerrit:576942|Include required url in mediawiki/client/error event (T246030)]] (duration: 01m 05s) |
[production] |
| 20:03 |
<jeh> |
add cloudvirt-wdqs100[123] to ceph host aggregate T221631 |
[openstack] |
| 20:02 |
<jeh> |
add new ceph host aggregate T225320 |
[openstack] |
| 19:58 |
<otto@deploy1001> |
Synchronized php-1.35.0-wmf.22/extensions/WikimediaEvents/modules/ext.wikimediaEvents/clientError.js: SWAT: [[gerrit:576931|Fix callback parameters for client error logging (T246030)]] (duration: 01m 07s) |
[production] |
| 19:53 |
<hauskatze> |
GitHub: extension-LiveTranslate mirror deleted refs. T196452 |
[releng] |
| 19:52 |
<shdubsh> |
restart logstash on logstash2005 -- testing field type mismatch mitigation |
[production] |
| 19:34 |
<James_F> |
Shutting off deployment-parsoid09 for T246395 |
[releng] |
| 19:23 |
<James_F> |
Rebooting deployment-puppetmaster04 |
[releng] |
| 18:43 |
<mutante> |
starting new DHCP servers to confirm they work and letting puppet immediately stop them again to clear systemd status |
[production] |
| 18:30 |
<mutante> |
notebook1003 - restarted nagios-nrpe-server |
[production] |
| 17:41 |
<addshore> |
stop item term rebuild at Q Q60345318 as I generate more lists (T219123) |
[production] |
| 16:50 |
<otto@cumin1001> |
conftool action : set/pooled=true; selector: dnsdisc=eventgate-analytics-external |
[production] |
| 16:49 |
<cmjohnson@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 16:45 |
<cmjohnson@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 16:05 |
<ottomata> |
destroying unused eventgate-main 'main' and eventgate-analytics 'analytics' helm releases - installed: false |
[production] |
| 16:02 |
<addshore> |
addshore@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/rebuildPropertyTerms.php --wiki=wikidatawiki --sleep 1 --batch-size=50 # T244115 |
[production] |