|
2017-01-30
§
|
| 18:04 |
<gehel> |
rolling restart of nginx and wdqs for updates |
[production] |
| 17:41 |
<Pchelolo> |
update RESTBase to cd2b5e019: staging |
[production] |
| 17:09 |
<legoktm@tin> |
Finished scap: Build l10n cache for linter (duration: 22m 43s) |
[production] |
| 17:05 |
<marostegui> |
Shutdown mysql and poweroff db2034 for maintenance - T156478 |
[production] |
| 16:46 |
<legoktm@tin> |
Started scap: Build l10n cache for linter |
[production] |
| 16:40 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Depool db2034 for maintenance - T156478 (duration: 00m 40s) |
[production] |
| 16:34 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=wdqs2003.codfw.wmnet |
[production] |
| 15:42 |
<hashar@tin> |
Synchronized php-1.29.0-wmf.9/extensions/timeline/Timeline.body.php: debug log EasyTimeline error - T138036 (duration: 00m 46s) |
[production] |
| 15:39 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1073 with its original weight - T156226 (duration: 00m 52s) |
[production] |
| 14:45 |
<hashar@tin> |
Synchronized php-1.29.0-wmf.9/languages/Language.php: translateBlockExpiry: Duration is block expiry minus current time - T156453 (duration: 00m 42s) |
[production] |
| 14:29 |
<hashar@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable RSS extension at metawiki, enable one feed - T155830 (duration: 00m 42s) |
[production] |
| 14:15 |
<hashar@tin> |
Synchronized wmf-config/InitialiseSettings.php: Create namespace alias وگ for NS_PROJECT in fawikiquote - T156451 (duration: 00m 40s) |
[production] |
| 14:10 |
<hashar@tin> |
Synchronized wmf-config/flaggedrevs.php: Remove flaggedrevs-protect-review page protection from enwiki - T156448 (duration: 00m 41s) |
[production] |
| 14:08 |
<hashar@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable SandboxLink on tgwiki - T156473 (duration: 00m 40s) |
[production] |
| 14:06 |
<hashar@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable SandboxLink on gdwiki - T156281 (duration: 00m 48s) |
[production] |
| 11:23 |
<gehel> |
upgrade and restart nginx on elasticsearch eqiad cluster |
[production] |
| 10:03 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1073 with less weight - T156226 (duration: 00m 49s) |
[production] |
| 10:00 |
<gehel> |
upgrade and restart nginx on elasticsearch codfw cluster |
[production] |
| 09:58 |
<gehel> |
upgrade and restart nginx on relforge cluster |
[production] |
| 09:44 |
<godog> |
upgrade to thumbor 0.1.33 - T151066 |
[production] |
| 09:37 |
<ariel@tin> |
Finished deploy [dumps/dumps@4a9e952]: proper md5sum format for adds/changes dumps (duration: 00m 02s) |
[production] |
| 09:37 |
<ariel@tin> |
Starting deploy [dumps/dumps@4a9e952]: proper md5sum format for adds/changes dumps |
[production] |
| 09:25 |
<elukey> |
bootstrapping new cassandra instance (aqs1007-b) on AQS - https://gerrit.wikimedia.org/r/#/c/334753/ |
[production] |
| 09:19 |
<moritzm> |
installing tcpdump security updates |
[production] |
| 09:06 |
<marostegui> |
Upgrade db2012 to 10.0.29-2 (this was done couple of hours ago, but for the record) - T156373 |
[production] |
| 09:05 |
<marostegui> |
Start slaves from s1 to s7 on dbstore2001 - T156373 |
[production] |
| 08:54 |
<moritzm> |
installing NSS security updates on kafka and Hadoop clusters |
[production] |
| 08:45 |
<elukey> |
restarting aqs on aqs100[4567] to pick up NSS updates |
[production] |
| 08:19 |
<elukey> |
set mw1236.eqiad.wmnet pooled=inactive because powered off (no mentions on the SAL, still trying to find why) |
[production] |
| 08:05 |
<moritzm> |
switched application servers in codfw to systemd-timesyncd |
[production] |
| 08:04 |
<marostegui> |
Stop mysql db1073 to use it to clone db1072 - T156226 |
[production] |
| 08:01 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1073 - T156226 (duration: 02m 45s) |
[production] |
| 02:21 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Mon Jan 30 02:21:21 UTC 2017 (duration 4m 22s) |
[production] |
| 02:16 |
<l10nupdate@tin> |
scap sync-l10n completed (1.29.0-wmf.9) (duration: 06m 11s) |
[production] |
|
2017-01-27
§
|
| 22:45 |
<mutante> |
install1001 - adding a second virtual hard disk, 80G |
[production] |
| 22:31 |
<mutante> |
carbon: rsync entire /srv/ to install2001 (this is APT data but also misc things like junos, megacli, firmware, ipmi |
[production] |
| 21:41 |
<volans> |
restored watchmouse checks for s5 (de wiki), Main_Page redirect was restored |
[production] |
| 21:36 |
<mobrovac@tin> |
Finished deploy [trending-edits/deploy@0e79bec]: Bump max_age to 12h T156411 (duration: 01m 58s) |
[production] |
| 21:34 |
<mobrovac@tin> |
Starting deploy [trending-edits/deploy@0e79bec]: Bump max_age to 12h T156411 |
[production] |
| 21:28 |
<mobrovac@tin> |
Finished deploy [trending-edits/deploy@e0e32bb]: Restart the service to assess the load of replaying the last 6h T156411 (duration: 01m 03s) |
[production] |
| 21:27 |
<mobrovac@tin> |
Starting deploy [trending-edits/deploy@e0e32bb]: Restart the service to assess the load of replaying the last 6h T156411 |
[production] |
| 20:24 |
<jynus> |
restart and upgrade mariadb on db1048 |
[production] |
| 20:14 |
<mutante> |
db1029, db1042, analytics1015, analytics1026 - puppet node deactivate, remove from icinga, finish decom (T147313, T149793, T146265) |
[production] |
| 20:07 |
<volans> |
updated watchmouse checks for s5 (de wiki) because Main_Page was deleted, used the localized page instead |
[production] |
| 19:52 |
<mutante> |
db1019 - shutdown -h now (T146265) |
[production] |
| 19:51 |
<mutante> |
db1042 - i came to shut it down .. and noticed it had died (or somebody did it) about 3 hours ago .. there it goes (T149793) |
[production] |