|
2019-02-28
ยง
|
| 20:52 |
<herron> |
cleared logstash persistent queue on logstash100[7-9] |
[production] |
| 20:13 |
<hashar@deploy1001> |
rebuilt and synchronized wikiversions files: all wikis to 1.33.0-wmf.19 |
[production] |
| 20:02 |
<thcipriani@deploy1001> |
Synchronized wmf-config/throttle.php: SWAT: [[gerrit:493425|Add throttle exception for Amnesty International Editathon]] [[gerrit:493496|Thottle Rules: remove "all"]] [[gerrit:493375|Add new throttle rules]] T216998 T217063 T217305 T217311 (duration: 00m 54s) |
[production] |
| 19:40 |
<thcipriani@deploy1001> |
Synchronized wmf-config/CommonSettings.php: SWAT [[gerrit:492770|Remove legacy eventBus config settings.]] (duration: 00m 53s) |
[production] |
| 19:36 |
<_joe_> |
upgrading scap on all servers |
[production] |
| 19:30 |
<thcipriani@deploy1001> |
Synchronized wmf-config/throttle.php: SWAT: [[gerrit:493427|Add throttle rule for Art+Feminism 2019 editathon]] T217336 (duration: 00m 54s) |
[production] |
| 19:26 |
<thcipriani@deploy1001> |
Synchronized wmf-config/InitialiseSettings-labs.php: SWAT: [[gerrit:493321|Enable WikibaseCirrusSearch on Beta Cluster]] (beta only change/noop sync) T215684 (duration: 00m 55s) |
[production] |
| 19:22 |
<robh> |
mw1272 being worked on by onsite |
[production] |
| 19:21 |
<robh> |
mw1272 unresponsive to mgmt or production interfaces |
[production] |
| 19:16 |
<thcipriani@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:493287|GrowthExperiments: Start help panel experiment on viwiki]] T215666 (duration: 03m 02s) |
[production] |
| 18:52 |
<moritzm> |
installing libgd security updates on trusty |
[production] |
| 18:52 |
<herron> |
migrating logstash1006 kafka to logstash1012 T213898 |
[production] |
| 18:43 |
<XioNoX> |
start pybal on lvs1016 - T212348 |
[production] |
| 18:34 |
<robh> |
cp1078 power down for network move |
[production] |
| 18:28 |
<XioNoX> |
stop pybal on lvs1016 - T212348 |
[production] |
| 18:28 |
<robh> |
cp1077 power off for network port relocation |
[production] |
| 18:21 |
<robh> |
cp1076 power down for network port move |
[production] |
| 17:51 |
<herron> |
logstash1011 kafka now in sync. transitioning logstash1005 to spare system T213898 |
[production] |
| 17:24 |
<cmjohnson1> |
powering down sodium to move racks T212348 |
[production] |
| 17:23 |
<jynus> |
recreating replicas, master ops events for db1078, db1075 T213858 |
[production] |
| 16:43 |
<filippo@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=ms-fe1006.eqiad.wmnet |
[production] |
| 16:39 |
<elukey> |
clean up old/stale zookeeper znodes from conf100[4-6] - T216979 |
[production] |
| 16:28 |
<herron> |
migrating kafka on logstash1005 to logstash1011 T213898 |
[production] |
| 16:27 |
<herron> |
migrating kafka on logstash1005 to logstash1011 T213898 |
[production] |
| 16:15 |
<addshore@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T214905 Add ReferencePreviews to allowed BetaFeatures (duration: 00m 54s) |
[production] |
| 16:08 |
<jbond42> |
rebooting labstore2003 |
[production] |
| 15:56 |
<thcipriani@deploy1001> |
Synchronized README: noop sync to test opcache-manager in scap 3.9.1-1 (duration: 00m 53s) |
[production] |
| 15:52 |
<jbond42> |
rebooting labsdb1004 |
[production] |
| 15:50 |
<thcipriani@deploy1001> |
Synchronized README: noop sync scap 3.9.1-1 (duration: 00m 52s) |
[production] |
| 15:49 |
<akosiaris@deploy1001> |
scap-helm citoid finished |
[production] |
| 15:49 |
<akosiaris@deploy1001> |
scap-helm citoid cluster staging completed |
[production] |
| 15:48 |
<akosiaris@deploy1001> |
scap-helm citoid upgrade -f citoid-staging-values.yaml staging stable/citoid [namespace: citoid, clusters: staging] |
[production] |
| 15:46 |
<_joe_> |
install scap 3.9.1-1 on the deployment servers |
[production] |
| 15:43 |
<filippo@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=ms-fe1006.eqiad.wmnet |
[production] |
| 15:43 |
<jbond42> |
rebooting labsdb1007 |
[production] |
| 15:37 |
<jbond42> |
rebooting labsdb1006 |
[production] |
| 15:36 |
<filippo@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=ms-fe1005.eqiad.wmnet |
[production] |
| 15:33 |
<jbond42> |
rebooting labstore2002 |
[production] |
| 15:29 |
<jbond42> |
rebooting labstore2001 |
[production] |
| 15:23 |
<filippo@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=ms-fe1005.eqiad.wmnet |
[production] |
| 15:19 |
<jbond42> |
rebooting rhodium |
[production] |
| 15:15 |
<cmjohnson1> |
powering off db1114 to replace motherboard T214720 |
[production] |
| 15:14 |
<_joe_> |
uploading scap 3.9.1-1 to {stretch,jessie}-wikimedia |
[production] |
| 14:50 |
<jbond42> |
reboot cloudnet2001-dev.codfw.wmnet |
[production] |
| 14:47 |
<hashar> |
mw1272 fixed by running "scap sync-l10n" from deploy host |
[production] |
| 14:46 |
<hashar> |
mw1272 had /srv/mediawiki/php-1.33.0-wmf.19/includes/cache/localisation/LocalisationCache.php:475) No localisation cache found for English. Please run maintenance/rebuildLocalisationCache.php. |
[production] |
| 14:46 |
<hashar@deploy1001> |
scap sync-l10n completed (1.33.0-wmf.19) (duration: 03m 33s) |
[production] |
| 14:42 |
<jbond@cumin1001> |
conftool action : set/pooled=no; selector: name=rhodium.eqiad.wmnet |
[production] |
| 14:41 |
<hashar@deploy1001> |
Synchronized php: group1 wikis to 1.33.0-wmf.19 (duration: 00m 53s) |
[production] |
| 14:40 |
<hashar@deploy1001> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.33.0-wmf.19 |
[production] |