Tip: 看不到本站引用 Flickr 的图片? 下载 Firefox Access Flickr 插件 | AD: 订阅 DBA notes -- 
2010-06-02 Wed
This run is very similar what I had on Intel SSD X25-M card, but now I use FusionIO 80GB SLC card. I chose this card as smallest available card (and therefore cheapest. On Dell.com you can see it for about $3K). There is also FusionIO IO-Xtreme 80GB card, which is however MLC based and it could be not best choice for FlashCache usage ( as there high write rate on FlashCache for both reading and writing to/from disks, so lifetime could be short).
Also Facebook team released WriteThrough module for FlashCache, which could be good trade-off if you want extra warranty for data consistency and your load is mostly read-bound, so I tested this mode also.
All setup is similar to previous post, so let me just post the results with FlashCache on FusionIO in 20% dirty page, 80% dirty pages and write-through modes. I used full 80GB for caching ( total size of data is about 100GB).
Conclusions from the graph:
- with 80% dirty page we have about 4x better throughput ( comparing to RAID).
- Write-through mode is about 2x gain, but remember that load is very write intensive and all benefits in write-through mode come only from cached reads, so it is pretty good for this scenario
On this post I finish my runs on FlashCache for now and I think it may be considered for real usage, at least you may evaluate how it works on your workloads.
Entry posted by Vadim | One comment
Add to: 
| 
| 
| 
| 
Finally I was able to run PBXT 1.0.11 pre-GA in tpcc-like workload, apparently there was bug with did not allow me to get the result earlier, and I am happy to see that PBXT team managed it.
For initial runs I took tpcc 100 warehouses ( about 10GB of data) which fully fits into memory (32 GB on server),
and compared 1 and 16 users in MySQL-5.1.46/PBXT and Percona Server / XtraDB – 5.1.45-rel10.2. As workload is totally memory based it will show how PBXT scales in CPU-bond cases on 16 cores systems.
As storage system it was Intel SSD X25-M card.
While full results and config are on Wiki:
http://www.percona.com/docs/wiki/benchmark:pbxt:tpcc:start
there are graphs for 1 user:
and 16 users:
Interesting to see that in case with 1 user the maximal throughput in PBXT is about 1.5x better XtraDB, but
there periodical drops which are very similar to periodical drops in InnoDB without adaptive checkpointing, and I guess it is also related to checkpoint activity.
The final results are also better for PBXT: 5785.567 TpmC vs 4905.967 TpmC ( XtraDB)
For 16 threads final result is: 26129.350 TpmC for PBXT and 29485.518 TpmC for XtraDB , and from the graph you can see that the maximal throughput is about identical, while PBXT spends more time in drops area. Again it looks like PBXT are not fully keeping up with checkpoint activity and I am looking PBXT addresses this problem also. Beside this issue PBXT looks pretty good and in next round I am going to run IO intensive workloads.
Entry posted by Vadim | 6 comments
Add to: | | | |
[yanick@enkidu shuck]$ perl -MWWW::Robot
my $robot = WWW::Robot->new(
NAME => 'shuck', VERSION => 0.1, EMAIL => 'blogger@pythian.com' );
$robot->addHook( 'follow-url-test' => sub { 1 } );
$robot->addHook( 'invoke-on-contents' => sub { print $_[5] if rand() > 0.5 } );
$robot->run( 'http://blogs.perl.com' );
^D
First, Inigo Tejedor reminds us that we have until Thursday June 3rd (yes, tomorrow) to fill out the Perl programming survey. If you haven’t done so already, what are you waiting for? Stop reading this blog entry right now and go do your duty. No, seriously, go!
And no peeking back until you’re done!
… so, survey’s filled out? Good. Now we can continue.
Marcus Ramberg echoes an IRC chat announcing that DBIx::Class is switching its official repository from svn to git.
Jared contests the claim that the choice of a language ultimately makes little difference.
Dave Rolsky presents his views on the advantages and pitfalls of using Dist::Zilla.
Ovid is showing us how to calculate the Roman date of a most happy event.
The results are in for the hair color and talk that Matt S. Trout will have to adopt as a penalty for failing the Perl Ironman contest. His hair will be dyed a pristine snow-white, and his talk will be about how Module::Build is better than ExtUtils::MakeMaker, and how does this reflect on the design of Perl 7 and the fact that PHP is the future of web development, which will tie in in the historic review of MVCs, from Pacman to Django, with mention that patches are indeed welcome, and that — by the by — ferrets are an excellent source of (ferrety) food, perhaps due to the fact that Apple is the devil incarnate, which is not a reason to use cognitive science to improve HTML::Zoom’s usability. It is a safe bet to say that the show is definitively going to be worth the admission price.
A silly little trick, but perhaps unknown to some: Andy Lester shows the easy way to shuffle a list in Perl.
Ashley Pond V has a success story about how Perl and DBIx::Class saved his AdSense account from the Puritan mob.
fREW Schmidt shares a quick trick on how to sync with multiple git repositories with a single git push all.
Curtis Jewell let us know that the new Strawberry July 2010 Beta 1 is out.
Paul Evans began with weakening references in his code, and ended up seeing weasels crawling all over his objects.
Adam Kennedy reports a most flattering quote coming from Rob Mensching, the creator of Windows Installer XML and the WiX toolkit.
David Wheeler does a little bit of PGAN bikeshedding. Which wouldn’t be that noteworthy until you realize that PGAN is a PostgreSQL extension distribution system based on CPAN — for anyone using PostgreSQL and knowing the power of CPAN, that should be enough to trigger a few dreamy “ooooooh”s.
You want to develop a modern Perl web application, but can’t find a hosting company that lets you have the environment to do so? Patrick Donelan might have a very enticing proposition for you with his shared hosting project.
[yanick@enkidu shuck]$ perl -E'sleep 2 * 7 * 24 * 60 * 60 # see y'all in 2 weeks!'
In Percona Server security fix releases I mentioned patches for MySQL 4.0 and 4.1.
I am happy to announce that GoDaddy.com released patches for MySQL 4.0 and MySQL 4.1 under GPL license and you can get them from our Launchpad:
for 4.0: lp:~percona-dev/percona-patches/4.0.30 ( or https://launchpad.net/~percona-dev/percona-patches/4.0.30 )
for 4.1: lp:~percona-dev/percona-patches/4.1.24 ( or https://launchpad.net/~percona-dev/percona-patches/4.1.24)
Fixed bugs:
If you are using MySQL 4.0 and 4.1 with public user access we strongly recommend to patch your MySQL instances. You need help with binaries you may contact us or wait until we have binaries public available.
Entry posted by Vadim | No comment
Add to: | | | |
☞ A simple notice about a short scheduled GitHub downtime resulted in a great dialog about how to perform zero downtime Redis upgrades. The following approaches were suggested:
- using Redis replication (i.e.
SLAVE OF) to move existing data from the old Redis to the new Redis and haproxy to transparently pass client requests to the Redis instance (initially to the existing version and once replication completed to the new one)- Leave the old redis version running on, say, redis:6379.
- Install and start a new redis on redis:6380 with a different dump file location.
- Execute SLAVE OF redis 6379 against redis:6380. Wait for first SYNC to complete.
echo "enable server redis/redis-6380" | socat stdio unix-connect:/var/run/haproxy/admin.sockecho "disable server redis/redis-6379" | socat stdio unix-connect:/var/run/haproxy/admin.sock- Execute SLAVE OF no one on redis:6380.
- Execute SHUTDOWN on redis:6379.
- using virtual IPs instead of haproxy
A more generic idea shared in the comment thread was to architect the solution so that the overall system would continue to work even if some subsystems are temporarily down. If you think in terms of the CAP theorem, this idea could be translated as a system that is AP at all times.
While writing this post I remembered reading about a ☞ solution based on node.js. node.js would be used to proxy requests until it is notified that the back system goes down. It would temporarily hold incoming connections until it would be notified again that the back system is resuming.
Any other ideas?
PYTHIAN NEWS UPDATE
The Pythian Group, Inc., a leading provider of remote database infrastructure services, is pleased to announce its selection as one of Canada’s fastest-growing companies. Measured against a running five-year revenue growth, Pythian is listed in the NEXT 100, making it one the top 200 companies overall in the 22nd annual PROFIT 100 ranking of Canada’s Fastest-Growing Companies by PROFIT Magazine.
“We are ecstatic to have our organization achieve the a position on the Profit 100, Next 100 list” said Andrew Waitman, Pythian’s Chief Executive Officer. “Our revenue growth is a true testament to our talented team of industry experts and the hours of hard work we put in for our clients.”
Paul Vallee, Pythian’s Founder and Executive Chairman attributes the success to the company’s unique, flexible outsourcing engagement model. “The world is different today. Rapid economic changes brought about by global economic forces require businesses to be agile, and able to rapidly respond to increases or decreases in demand. Service provider models must also adapt to the needs and demands of today’s customer.”
Over 120 companies globally rely on Pythian as their trusted services partner to plan, deploy, and manage their complex database infrastructures, and for its expertise in Oracle, Oracle Applications, MySQL and SQL Server. The company provides database, systems administration and emergency response services for projects of any scale, to partners and clients, ranging from midsize businesses to the Global 2000, across key verticals such as online media, telecommunications, finance, and healthcare. Its flexible, month-to-month utility-style billing model has proven to be a cost-effective approach to managing databases; many of which are mission critical to these organizations.
Ranking Canada’s Fastest-Growing Companies by five-year revenue growth, the PROFIT 100 profiles the country’s most successful growth companies. Published in the June issue of PROFIT and online at PROFIT100.com, the PROFIT 100 is Canada’s largest annual celebration of entrepreneurial achievement.
“Canada’s Fastest-Growing Companies are the poster children for entrepreneurship, innovation and pure tenacity,” says Ian Portsmouth, editor of PROFIT. “They demonstrate what it takes to succeed in today’s highly competitive, technology-driven global economy.”
About Pythian
Pythian is a trusted global database and application infrastructure services company for Oracle, MySQL and SQL Server. Since 1997, companies have entrusted Pythian to keep their database infrastructures running efficiently while assisting organizations in strategically aligning their IT with business goals. Pythian’s unparalleled DBA skills, mature methodologies, best practices and tools enable clients to do more with fewer resources. Pythian’s corporate headquarters is in Ottawa Canada, with offices worldwide. To find out more visit http://www.pythian.com
About PROFIT Magazine:
PROFIT: Your Guide to Business Success is Canada’s preeminent publication dedicated to the management issues and opportunities facing small and mid-sized businesses. For 28 years, Canadian entrepreneurs and senior manag¬ers across a vast array of economic sectors have remained loyal to PROFIT because it’s a timely and reliable source of actionable information that helps them achieve business success and get the recognition they deserve for generat¬ing positive economic and social change. Visit PROFIT online at www.profitmagazine.ca.
For Media inquiries contact: Vanessa Simmons, Director of Marketing, simmons@pythian.com or P: 613.565.8696 ext. 208/C: 613.897.9444
Not sure how many smalltalkers are reading the NoSQL blog and how many of them are interested in CouchDB, but this was too geeky to resist it:
Setting up a Squeak+Seaside+CouchDB dev environment: This is a little log from my attempts to set up a fresh development environment for the eBlankett.org project. eBlankett is a web system that presents wizard like web forms to the user given a declarative high level definition of this wizard in JSON.
Vasil Remeniuk shares some tricks on working with MongoDB JavaScript:
This approach (nb: inlined JavaScript ) has two obvious drawbacks:
- Performance. Evaluated functions are not persisted between connects, therefore should be re-evaluated, which takes a lot of time (especially, if you deal with complex calls);
- Testability and readability. In-lined code is untestable and looks ugly.
[…]
First of all, all the in-lined JavaScript code should be moved to the place, where it should be – .js files on the classpath that can be tested (and mocked), and recognized by IDEs, which make them more comfortable to edit (because oh highlights, JavaDocs, autocompletion and other features we love our IDEs for).
When analyzing how good or bad response time is it is not handy to look at the averages, min or max times - something what is easily computed using built in aggregate functions. We most likely would like to see some percentile numbers - 95 percentile or 99 percentile. The problem is computing these in SQL is ugly and resource consuming. There is however a simple way to get similar data, looking at it from the different point of view.
When we're speaking about application we may not always care about exact value of response time but rather we want to see response time to be within certain range. For example if we define page feels good if response time is below 50ms it is not as important if response was 40ms or 44ms - it is much more important how frequently this goal was reached.
In fact I prefer to define 2 performance level. One what users would consider good performance, another what is acceptable, and basically the third level is unacceptable. We can also set classical color codes to them - green, yellow, red.
For example we can define good performance as response time for search pages within 1 second, acceptable as response within 3 seconds and anything over that will be considered unacceptable:
-
mysql> SELECT sum(wtime<1.0)/count(*)*100 green_pct, sum(wtime>=1.0 AND wtime<3.0)/count(*)*100 yellow_pct, sum(wtime>=3.)/count(*)*100 red_pct FROM performance_log_100601 WHERE page_type='search';
-
+-----------+------------+---------+
-
| green_pct | yellow_pct | red_pct |
-
+-----------+------------+---------+
-
| 99.6701 | 0.3006 | 0.0293 |
-
+-----------+------------+---------+
-
1 row IN SET (24.87 sec)
These are great numbers to look after. In this case we can say system has "two nines" in terms of performance - it responds with what we consider "good" performance in over 99% cases. I consider this a lot more meaningful number for business than speaking about 99% response time, especially as it also clearly clarifies how bad the problem is. If your goal is 99% response time is 1sec and you're seeing it 2 seconds what you really do not know is how many people fall in this 1-sec to 2 sec range. It could be very small number and so your performance is not that bad or it could be majority of response times are withing 1 to 2 sec range.
Next thing I would add is the average response time for our good requests. This eliminates slow outliers which we just count by their number and allows to see how good our performance really is. I consider it secondary value as our major goal is eliminating variance as defined by Six Sigma practices. However given variance is the same better average response time for good requests helps us to track performance changes closely and act before we get large number of yellow or red requests:
-
mysql> SELECT avg(IF(wtime<1.0,wtime,NULL)) avg_green,sum(wtime<1.0)/count(*)*100 green_pct, sum(wtime>=1.0 AND wtime<3.0)/count(*)*100 yellow_pct, sum(wtime>=3.0)/count(*)*100 red_pct FROM performance_log_100601 WHERE page_type='search';
-
+-------------------+-----------+------------+---------+
-
| avg_green | green_pct | yellow_pct | red_pct |
-
+-------------------+-----------+------------+---------+
-
| 0.448726461517477 | 99.6701 | 0.3006 | 0.0293 |
-
+-------------------+-----------+------------+---------+
-
1 row IN SET (6.84 sec)
Finally I find it helpful to track it among time period. For example in this case we're looking for data grouped by 15 min interval though you may be looking at different intervals too:
-
mysql> SELECT from_unixtime(unix_timestamp(logged) div 900 * 900) period, avg(IF(wtime<1.0,wtime,NULL)) avg_green,sum(wtime<1.0)/count(*)*100 green_pct, sum(wtime>=1.0 AND wtime<3.0)/count(*)*100 yellow_pct, sum(wtime>=3.0)/count(*)*100 red_pct FROM performance_log_100601 WHERE page_type='search' GROUP BY 1;
-
+---------------------+-------------------+-----------+------------+---------+
-
| period | avg_green | green_pct | yellow_pct | red_pct |
-
+---------------------+-------------------+-----------+------------+---------+
-
| 2010-05-31 20:00:00 | 0.454738099705166 | 100.0000 | 0.0000 | 0.0000 |
-
| 2010-05-31 20:15:00 | 0.477789820622135 | 100.0000 | 0.0000 | 0.0000 |
-
| 2010-05-31 20:30:00 | 0.463134796179216 | 100.0000 | 0.0000 | 0.0000 |
-
| 2010-05-31 20:45:00 | 0.397884420995359 | 99.4595 | 0.5405 | 0.0000 |
-
| 2010-05-31 21:00:00 | 0.438939880065877 | 100.0000 | 0.0000 | 0.0000 |
-
| 2010-05-31 21:15:00 | 0.48698966072074 | 100.0000 | 0.0000 | 0.0000 |
-
| 2010-05-31 21:30:00 | 0.432073892663709 | 93.6364 | 6.3636 | 0.0000 |
-
| 2010-05-31 21:45:00 | 0.483174517098134 | 100.0000 | 0.0000 | 0.0000 |
-
| 2010-05-31 22:00:00 | 0.449507013872618 | 100.0000 | 0.0000 | 0.0000 |
-
| 2010-05-31 22:15:00 | 0.327130372130607 | 100.0000 | 0.0000 | 0.0000 |
-
| 2010-05-31 22:30:00 | 0.431361319540545 | 100.0000 | 0.0000 | 0.0000 |
-
| 2010-05-31 22:45:00 | 0.480885540467242 | 100.0000 | 0.0000 | 0.0000 |
Note response time does not always correlates with number of bad requests. For example we can see at one interval there were over 6% requests in yellow but average response time for good requests actually was better than in the other intervals.
The queries which produce this data may not be overly beautiful but they are pretty fast requiring simple scan of the data range.
P.S Gathering data this way also often gives you some idea about what your percentile response time numbers might be. In this case I know over 99% of requests are served in 1sec so I know my 99% response time is below 1 second.
Entry posted by peter | 4 comments
Add to: | | | |
2010-06-01 Tue
AnySQL.net
Oracle & Starcraft
Give you some color to see see!
Oracle Scratchpad
Oracle Life
Chanel [K]
Oracle Security Blog
MySQL Performance Blog
The Tom Kyte Blog
Delicious/Fenng/oracle
O'Reilly Databases
Red Hat Magazine
车东[Blog^2]
blue_prince
玉面飞龙的BLOG
木匠 Creative and Flexible
生活帮-LifeBang
Hey!! Sky!
dba on unix
Brotherxiao's Home
jametong's shared items in Google Reader
DBA Tools
Inside the Oracle Optimizer - Removing the black magic
DBA@Taobao
存储部落
OracleBlog.cn
知道分子
支付宝官方 Blog - 支付志
木匠的天空 Oracle Architect and Developer
Hello DBA
OS与Oracle
Cary Millsap
Guy Harrison's main page
eagle's home
dbthink
OracleDBA Blog---三少个人涂鸦地！The Pythian Blog
myNoSQL
- OracleDBA Blog---三少个人涂鸦地！
DBA@SKY-MOBI