|
2016-09-01
ยง
|
| 20:58 |
<mutante> |
mw2187 - shut down |
[production] |
| 20:35 |
<urandom> |
T143226: Clearing repair status: eqiad, rack 'd' nodes |
[production] |
| 20:35 |
<urandom> |
T143226: Clearing repair status: eqiad, rack 'dd' nodes |
[production] |
| 19:58 |
<hashar> |
1.28.0-wmf.17 rolled on group2 and apparently all fine |
[production] |
| 19:54 |
<gehel> |
reloading ferm rules on elasticsearch eqiad cluster |
[production] |
| 19:44 |
<urandom> |
T143226: Clearing repair status: eqiad, rack 'b' nodes |
[production] |
| 19:39 |
<urandom> |
T143226: Clearing repair status restbase1011-c.eqiad.wmnet |
[production] |
| 19:19 |
<hashar@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.28.0-wmf.17 |
[production] |
| 19:14 |
<mutante> |
tin temp. disabled puppet |
[production] |
| 19:11 |
<mutante> |
tin removing mw2167 thru mw2199 from dsh file manually, re-running puppet |
[production] |
| 18:33 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:307927|Test PageAssessments on English Wikivoyage (T142056)]] (duration: 02m 48s) |
[production] |
| 18:26 |
<mutante> |
mw2187 - powercycled |
[production] |
| 18:19 |
<gehel@palladium> |
conftool action : set/pooled=yes; selector: name=elastic1029.eqiad.wmnet |
[production] |
| 18:19 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: REVERT because proxy down SWAT: [[gerrit:307927|Test PageAssessments on English Wikivoyage (T142056)]] (duration: 03m 15s) |
[production] |
| 18:11 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:307927|Test PageAssessments on English Wikivoyage (T142056)]] (duration: 04m 54s) |
[production] |
| 17:55 |
<Jeff_Green> |
switching fundraising database reader from db1008 to frdb1001 |
[production] |
| 17:12 |
<bd808> |
Updated striker to ac555bd; fixes T144064 |
[production] |
| 16:49 |
<ema> |
cp4006 repooled after downgrade |
[production] |
| 16:44 |
<gehel@palladium> |
conftool action : set/pooled=yes; selector: name=elastic1028.eqiad.wmnet |
[production] |
| 16:24 |
<ema> |
restarting pybal on lvs4002 T134893 |
[production] |
| 16:09 |
<ema> |
downgrading cp4006 to varnish 3 T131502 |
[production] |
| 15:36 |
<gehel@palladium> |
conftool action : set/pooled=yes; selector: name=elastic102[12].eqiad.wmnet |
[production] |
| 15:34 |
<chasemp> |
reboot nova-compute on labvirt1013 as stuck (no logs, not applying any changes or taking any instruction) |
[production] |
| 14:58 |
<moritzm> |
powered down several hosts for hardware maintenance (T142726): mw2099, mw2102, mw2117, mw2163-mw2199 |
[production] |
| 14:42 |
<mobrovac> |
restbase deploy end of 9cca320 |
[production] |
| 14:40 |
<moritzm> |
powered down several hosts for hardware maintenance (T142726): mw2087, mw2149-mw2151 |
[production] |
| 14:39 |
<elukey> |
upgrading httpd/apache to 2.4.10-10+deb8u6+wmf2 on mw128[56] |
[production] |
| 14:27 |
<mobrovac> |
restbase deploy start of 9cca320 |
[production] |
| 14:11 |
<godog> |
wipe and reinitialize corrupted xfs on /dev/sdn1 on ms-be1016 |
[production] |
| 13:53 |
<eileen> |
turned off queue for http://localhost:9000/job/GlobalCollect%20Recurring%20Donations/ in jenkins |
[production] |
| 13:42 |
<elukey> |
upgrading httpd/apache to 2.4.10-10+deb8u6+wmf2 on mw128[78] |
[production] |
| 13:41 |
<bblack> |
uploaded openssl-1.1.0-1+wmf1 to jessie-wikimedia/experimental |
[production] |
| 13:39 |
<elukey> |
not upgrading mw130[01] since I'd need more info before proceeding |
[production] |
| 13:34 |
<hashar@tin> |
Synchronized php-1.28.0-wmf.17/includes/page/WikiPage.php: T144484 (duration: 00m 49s) |
[production] |
| 13:30 |
<hashar@tin> |
Synchronized php-1.28.0-wmf.17/includes/page/WikiPage.php: T144484 (duration: 00m 35s) |
[production] |
| 13:16 |
<elukey> |
upgrading httpd/apache to 2.4.10-10+deb8u6+wmf2 on mw130[01].eqiad.wmnet |
[production] |
| 13:01 |
<gehel@palladium> |
conftool action : set/pooled=yes; selector: name=elastic1047.eqiad.wmnet |
[production] |
| 12:53 |
<ema@palladium> |
conftool action : set/pooled=yes; selector: cp4005.ulsfo.wmnet (tags: ['dc=ulsfo', 'cluster=cache_upload', 'service=varnish-be']) |
[production] |
| 12:12 |
<gehel> |
rolling restart of ferm on elasticsearch eqiad cluster to account for moved servers - T143685 |
[production] |
| 10:49 |
<moritzm> |
installing libidn security updates |
[production] |
| 10:39 |
<godog> |
reboot ms-be1016, stuck again |
[production] |
| 10:24 |
<hashar@tin> |
Synchronized docroot/noc/index.html: link to conftool and wikitech pages on https://noc.wikimedia.org/ (duration: 00m 47s) |
[production] |
| 10:17 |
<gehel> |
repooled elastic104[456] - T144450 |
[production] |
| 09:58 |
<jynus> |
adding marostegui to wmf and ops on wikitech LDAP |
[production] |
| 09:57 |
<gehel@palladium> |
conftool action : set/pooled=inactive; selector: elastic1046.eqiad.wmnet |
[production] |
| 09:57 |
<gehel@palladium> |
conftool action : set/pooled=inactive; selector: elastic1045.eqiad.wmnet |
[production] |
| 09:56 |
<gehel@palladium> |
conftool action : set/pooled=inactive; selector: elastic1044.eqiad.wmnet |
[production] |
| 09:51 |
<moritzm> |
reimaging mw2200-2203 to jessie |
[production] |
| 09:37 |
<moritzm> |
reimaging mw2061-2064 to jessie |
[production] |
| 09:35 |
<elukey> |
reimaging mw2167 -> mw2170 to Jessie |
[production] |