|
2019-09-13
§
|
| 09:24 |
<gehel> |
deny access to /geoline on maps1004 - T232817 |
[production] |
| 09:11 |
<oblivian@puppetmaster1001> |
conftool action : set/pooled=true; selector: dnsdisc=kartotherian,name=eqiad |
[production] |
| 09:08 |
<godog> |
downtime kartotherian pages for 1h in codfw |
[production] |
| 09:01 |
<oblivian@puppetmaster1001> |
conftool action : set/pooled=inactive; selector: name=elastic1046.eqiad.wmnet |
[production] |
| 09:00 |
<oblivian@puppetmaster1001> |
conftool action : set/pooled=inactive; selector: name=elastic1017.eqiad.wmnet |
[production] |
| 08:57 |
<oblivian@puppetmaster1001> |
conftool action : set/pooled=false; selector: dnsdisc=kartotherian,name=eqiad |
[production] |
| 08:52 |
<godog> |
downtime kartotherian pages for 1h |
[production] |
| 08:48 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.ipmi-password-reset (exit_code=0) |
[production] |
| 08:48 |
<jmm@cumin2001> |
Updating IPMI password on 1 hosts - jmm@cumin2001 |
[production] |
| 08:47 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.ipmi-password-reset |
[production] |
| 08:47 |
<jmm@cumin2001> |
END (FAIL) - Cookbook sre.hosts.ipmi-password-reset (exit_code=99) |
[production] |
| 08:47 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.ipmi-password-reset |
[production] |
| 08:45 |
<gehel> |
stop tilerator on maps to help reduce load |
[production] |
| 08:37 |
<_joe_> |
rolling restart of karotherian |
[production] |
| 08:33 |
<_joe_> |
restarting kartotherian on maps1003, all workers seem stuck |
[production] |
| 05:58 |
<oblivian@deploy1001> |
Synchronized w/fatal-error.php: Adding core dump function to fatal-error (duration: 01m 04s) |
[production] |
| 05:40 |
<_joe_> |
live-hacking mw1348, setting rlimit_core = unlimited to allow core dumps to be taken |
[production] |
| 05:17 |
<effie> |
Rolling restart php-fpm across the fleet for 536400 |
[production] |
| 04:53 |
<vgutierrez> |
restarting ats-tls on cp4021 and cp2002 to pick up the new SSL session cache timeout - T231849 |
[production] |
| 04:50 |
<eileen> |
process-control config revision is 43a2677bcf - turned off gender import |
[production] |
| 04:29 |
<bd808> |
Updated to a468c10 (Point at new Cloud VPS puppetmaster) |
[tools.openstack-browser] |
| 02:23 |
<eileen> |
civicrm revision changed from c5ab5aea9e to 45dbfdb96f, config revision is 1da8391a9a |
[production] |
| 01:09 |
<XioNoX> |
add IPv6 sampling to cr1-eqiad |
[production] |
| 01:07 |
<XioNoX> |
enable netflow sampling on cr2-codfw |
[production] |
|
2019-09-12
§
|
| 23:37 |
<James_F> |
Zuul: Define pipelines for REL1_34 T232025 |
[releng] |
| 23:35 |
<XioNoX> |
enable netflow sampling on cr1-codfw |
[production] |
| 23:21 |
<urandom> |
decommissioning Cassandra, restbase2009-b -- T224553 |
[production] |
| 23:19 |
<jforrester@deploy1001> |
Synchronized wmf-config/CommonSettings.php: T223602 Read config from JSON, not serialised PHP on testwiki (duration: 01m 03s) |
[production] |
| 23:18 |
<jforrester@deploy1001> |
Synchronized multiversion/MWConfigCacheGenerator.php: T223602 Add ability to read config from JSON, not serialised PHP (duration: 01m 04s) |
[production] |
| 23:10 |
<eileen> |
process-control config revision is 1da8391a9a |
[production] |
| 22:53 |
<ayounsi@cumin2001> |
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) |
[production] |
| 22:48 |
<ayounsi@cumin2001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 22:43 |
<ayounsi@cumin2001> |
END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) |
[production] |
| 22:43 |
<ayounsi@cumin2001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 22:20 |
<XenoRyet> |
payments-wiki updated from 4ebbdb247d to 1f556670cf |
[production] |
| 22:19 |
<hashar> |
integration-slave-jessie-1002: rm /srv/pbuilder/build/* /srv/pbuilder/aptcache/*/* |
[releng] |
| 22:16 |
<hashar> |
Pooling back integration-slave-jessie-1002 in Jenkins ( /srv was 95% full ) |
[releng] |
| 22:15 |
<hashar> |
integration-slave-jessie-1002: apt-get clean |
[releng] |
| 22:14 |
<XioNoX> |
remove extra prepend in AMS-IX |
[production] |
| 21:45 |
<James_F> |
Temporarily taking deployment-deploy01 offline as CI is stuck |
[releng] |
| 21:20 |
<bd808> |
Forcing puppet run on ores-web-04.ores.eqiad.wmflabs to see if I can fix ssh access for halfak |
[ores] |
| 21:18 |
<hashar@deploy1001> |
Synchronized php-1.34.0-wmf.22/includes/libs/rdbms/lbfactory/LBFactoryMulti.php: Hardcode posix signal and log coredump - T232613 (duration: 01m 04s) |
[production] |
| 21:17 |
<mbsantos@deploy1001> |
Finished deploy [tilerator/deploy@5996843]: Deploy tilerator 1.1.4-wmf.0 (duration: 03m 18s) |
[production] |
| 21:14 |
<mbsantos@deploy1001> |
Started deploy [tilerator/deploy@5996843]: Deploy tilerator 1.1.4-wmf.0 |
[production] |
| 21:13 |
<mbsantos@deploy1001> |
Finished deploy [kartotherian/deploy@c4c9e8b]: Deploy kartotherian 1.1.4-wmf.0 (duration: 03m 52s) |
[production] |
| 21:09 |
<mbsantos@deploy1001> |
Started deploy [kartotherian/deploy@c4c9e8b]: Deploy kartotherian 1.1.4-wmf.0 |
[production] |
| 21:00 |
<urandom> |
decommissioning Cassandra, restbase2009 -- T224553 |
[production] |
| 20:54 |
<James_F> |
Zuul: Drop operations-mw-config-hhvm-composer-test-docker, the PHP72 one suffices |
[releng] |
| 20:48 |
<phamhi> |
Deleted tools-puppetdb-01.tools as it is no longer in used |
[tools] |
| 20:46 |
<bd808> |
Restarting matterbridge. Bot AWOL from channels |
[tools.bridgebot] |