|
2019-03-12
§
|
| 11:35 |
<arturo> |
delete wrong stretch-wikimedia `thirdparty` component in install1002 |
[production] |
| 11:12 |
<zeljkof> |
EU SWAT finished |
[production] |
| 11:11 |
<kartik@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:495842]] Add campaign prefix for EG tag (T216123) (duration: 00m 49s) |
[production] |
| 11:11 |
<moritzm> |
upgrading API servers/job runners servers in eqiad to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) (T216712) |
[production] |
| 10:32 |
<marostegui> |
Deploy schema change on db1082, lag will happen on s5 on labs |
[production] |
| 10:29 |
<gtirloni> |
re-enabled puppet on serpens and seaborgium |
[production] |
| 10:18 |
<gtirloni> |
updated slapd to version 2.4.47 on seaborgium (T217280) |
[production] |
| 10:17 |
<moritzm> |
upgrading API servers/job runners servers in codfw to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) (T216712) |
[production] |
| 10:14 |
<gtirloni> |
upgrading seaborgium to slapd 2.4.47 |
[production] |
| 09:39 |
<jynus> |
stop db1114 and restart it empty |
[production] |
| 09:25 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1082 (duration: 00m 48s) |
[production] |
| 08:57 |
<elukey> |
restart memcached on mc1019 to apply new settings - T217731 |
[production] |
| 08:50 |
<ema> |
cp1077 depooled again T217893 |
[production] |
| 08:49 |
<ema@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=cp1077.eqiad.wmnet |
[production] |
| 08:48 |
<moritzm> |
upgrading app servers in codfw to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) (T216712) |
[production] |
| 08:48 |
<ema> |
restart varnish-be on cp1077 T217893 |
[production] |
| 08:47 |
<moritzm> |
upgrading app servers in codfw to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) |
[production] |
| 08:46 |
<ema> |
cp1077 repooled T217893 |
[production] |
| 08:46 |
<ema@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=cp1077.eqiad.wmnet |
[production] |
| 08:36 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1082 for schema change (duration: 00m 48s) |
[production] |
| 08:34 |
<jynus> |
deploy core replica events to db1118 |
[production] |
| 08:15 |
<ema> |
cp1099: ferm.service failed to resolve prometheus1003.eqiad.wmnet. ferm restarted T202966 |
[production] |
| 07:18 |
<marostegui> |
Deploy schema change on db2052 (s5 codfw master), this will generate lag on codfw T71127 T51199 |
[production] |
| 07:17 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1113 after schema change and upgrade (duration: 00m 49s) |
[production] |
| 07:09 |
<marostegui> |
Upgrade mysql and kernel on db1113 |
[production] |
| 06:40 |
<kart_> |
Finished manual run of unpublished ContentTranslation draft purge script (T217818) |
[production] |
| 06:17 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1113 for schema change and upgrade (duration: 00m 50s) |
[production] |
| 05:20 |
<rxy> |
Flags +AV were set on cohaf in #cvn-simplewikis. |
[cvn] |
| 05:18 |
<rxy> |
Flags +AV were set on Tulsi in #cvn-sw. |
[cvn] |
| 04:04 |
<kart_> |
Started manual run of unpublished ContentTranslation draft purge script (T217818) |
[production] |
| 03:46 |
<bd808> |
Restarted uwsgi-invisible-unicorn on proxy-01 |
[project-proxy] |
| 03:21 |
<bd808> |
Removed redis sets with no record in the backing database (T133554) |
[project-proxy] |
| 03:00 |
<bd808> |
Fixed domains with trailing dot (T218064) |
[project-proxy] |
| 02:40 |
<ejegg> |
updated payments-wiki from f1a89d7045 to 7a312e371a |
[production] |
| 01:13 |
<bd808> |
Deleted dangling backend records in /etc/dynamicproxy-api/data.db (T218064) |
[project-proxy] |
| 00:22 |
<bd808> |
Switched *.tiles.wmflabs.org proxies to point to http://172.16.5.154:80 (T217992) |
[maps] |
|
2019-03-11
§
|
| 18:46 |
<Joan> |
Restarted CVNBot18 (Last message was received on RCReader 29613.939513 seconds ago). |
[cvn] |
| 18:09 |
<gtirloni> |
restarted shinken to clear invalid alerts |
[shinken] |
| 17:55 |
<addshore@deploy1001> |
Synchronized wmf-config/interwiki-labs.php: BETA ONLY https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/495723/ (duration: 00m 48s) |
[production] |
| 17:47 |
<bd808> |
Deployed 6b6d5cc Explicitly cast numeric data to int |
[tools.sge-status] |
| 17:43 |
<addshore@deploy1001> |
Synchronized wmf-config/InitialiseSettings-labs.php: BETA ONLY https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/495721/ (duration: 00m 49s) |
[production] |
| 17:23 |
<arturo> |
T215605 copy python-oath from jessie-wikimedia/thirdparty to stretch-wikimedia/thirdpary in reprepro |
[production] |
| 17:02 |
<jbond@cumin1001> |
END (FAIL) - Cookbook sre.hosts.ipmi-password-reset (exit_code=99) |
[production] |
| 17:02 |
<jbond@cumin1001> |
START - Cookbook sre.hosts.ipmi-password-reset |
[production] |
| 16:31 |
<jbond@cumin1001> |
END (FAIL) - Cookbook sre.hosts.ipmi-password-reset (exit_code=99) |
[production] |
| 16:31 |
<jbond@cumin1001> |
START - Cookbook sre.hosts.ipmi-password-reset |
[production] |
| 16:20 |
<bd808> |
Updated to dbb5f60 Guard against malformed accounting lines |
[tools.grid-jobs] |
| 15:59 |
<Krenair> |
previous !log was for deployment-prep |
[releng] |
| 15:56 |
<Krenair> |
added MSantos to projectadmin, per chat in -infrastructure he'll use it to work on maps. user is a new foundation software engineer |
[releng] |
| 15:53 |
<bd808> |
Manually started `service gridengine-master` on tools-sgegrid-master after reboot (T218038) |
[tools] |