2012-01-27 Fri

22:33 MapReduce With Hadoop: What Happens During Mapping (1991 Bytes) » myNoSQL

MapReduce With Hadoop: What Happens During Mapping:

An interesting look at what happens during the map phase in Hadoop and the impact of emitting key-value pairs:

a direct negative impact on the map time and CPU usage, due to more serialization

an indirect negative impact on CPU due to more spilling and additional deserialization in the combine step

a direct impact on the map task, due to more intermediate files, which makes the final merge more expensive

The main point of the dynaTrace blog post is that even if Hadoop makes it easy to throw more hardware at a problem, wasting resources with bad code in MapReduce tasks comes with a noticeable and measurable cost.

Original title and link: MapReduce With Hadoop: What Happens During Mapping (NoSQL database©myNoSQL)

21:42 Analysts' Predictions for Hadoop Market (2340 Bytes) » myNoSQL

Analysts' Predictions for Hadoop Market:

With so many players in the market^[1], it’s easy to see that not all of them will flourish. IDC has predicted that this year will see a lot of merger and acquisition activity as large technology companies rush to buy smaller companies with expertise in big data. By 2015, the analysts say it’s likely that none of the current “major players” in the Hadoop market will still exist.

These predictions have also a dark scary side. Not in the sense that existing companies that bring value to the market do not deserve good exits in the next 3-4 years. But most of the time, if not ignored, these statements will lead to an applification of BS and the creation of a ton of copy-cats bringing no value to a market that still has to see a lot of innovation, adoption, and return on investment for the users.

According to Benjamin Woo, program vice president for worldwide storage systems at IDC, there are over 200 companies that claim to be in the big data space. ↩

Original title and link: Analysts’ Predictions for Hadoop Market (NoSQL database©myNoSQL)

19:30 Install GI and DB PSU 11.2.0.2.5 Failed in VirtualBox (9287 Bytes) » Channel [K]

Oracle的Apply Patchset的方法一直是为人诟病的，其实步骤复杂倒也罢了，怕的是Oracle总在不停地修改Apply Patch的方法，Oracle的原意是让Apply Patch的语法越来越简单，但是各种各样的Patch，各种不同的命令，特别是很大的Bundle Patch，如果不仔细阅读Readme，千万不要轻易出手。

这次尝试在自己的VirtualBox虚拟机OEL6中给之前安装的GI（Oracle Restart）+ ASM + Oracle Database安装最新的11.2.0.2.5 PSU，遇到各种问题。

1. Patch解压的目录必须是grid用户和oracle用户拥有写权限的，如果没有写权限，会报错：

Opatch version check failed for oracle home  /u01/app/oracle/product/11.2.0/dbhome_1
Opatch version  check failed
update the opatch version for the failed homes and retry

安装需求是使用root用户来安装（这是我第一次看到在安装PSU的时候要求使用root用户），而我的虚拟机中由于没有足够的磁盘空间，所以将Mac中的下载目录作为Shared Folder映射到虚拟机中，因此改目录的属主是root，用户组是vboxsf，而且并不允许使用chmod直接修改。因此出现了权限问题。我的解决方法是将grid用户和oracle用户都加入vboxsf组中。

建议：在真实环境中，Patch解压目录应该属于dba用户组。

2. 我的Patch是解压在/media/sf_PSU目录下，解压以后生成了p13343447_112020_Linux-x86-64目录，其下有两个目录分别是13343424（这是DB PSU）和13343447（这是GI PSU），整个目录结构如下所示：

 |-media
 |--sf_PSU
 |---p13343447_112020_Linux-x86-64
 |-----13343424
 |-----13343447

按照Readme文档中描述的，opatch的命令应该写为：

opatch auto <UNZIPPED_PATCH_LOCATION>

此处的UNZIPPED_PATCH_LOCATION按照文档描述应该就是/media/sf_PSU目录，因为这是解压目录，但是实际上这份文档是有问题的，如果opatch命令写为：

opatch auto /media/sf_PSU -ocmrf /home/grid/ocm.rsp

其中的-ocmrf是另外一个问题，这个OCM的配置文件，根据Readme文档中描述的方法创建即可。

运行以上命令会报错：

Opatch version check failed for oracle home  /u01/app/oracle/product/11.2.0/dbhome_1
Opatch version  check failed
update the opatch version for the failed homes and retry

是的，你没有看错，我也没有贴错，确实报了一模一样的错误（虽然这两个错误都完全不是opatch版本的问题），所以，opatch的报错信息是不可信的，我们必须要去提示的log文件中仔细查看最后的错误信息。

 ZOP-49: Not able to execute the prereq. OPatch cannot inform if the patch satisfies minimum version requirement.
 PatchObject constructor: Input file "/media/sf_PSU/p13343447_112020_Linux-x86-64/etc/config/actions" or "/media/sf_PSU/p13343447_112020_Linux-x86-64/etc/config/inventory" does not exist.

因此，正确的opatch命令应该是：

opatch auto /media/sf_PSU/p13343447_112020_Linux-x86-64 -ocmrf /home/grid/ocm.rsp

3. Oracle软件所在的文件系统剩余空间必须要大于3G，如果不足，会报错：

patch /media/sf_PSU/p13343447_112020_Linux-x86-64/13343447  apply  failed  for home  /u01/app/grid/product/11.2.0/grid
ACFS-9459: ADVM/ACFS is not supported on this OS version: 'error: file /etc/SuSE-release: No such file or directory

可以看到，又是一次很无稽的报错信息，/etc/SuSE-release？拜托，这里只有/etc/redhat-release。

那么，仔细检查log文件，会发现如下的报错：

 Prerequisite check "CheckSystemSpace" failed.
 The details are:
 Required amount of space(3154696080) is not available.
 UtilSession failed: Prerequisite check "CheckSystemSpace" failed.
 Log file location: /u01/app/grid/product/11.2.0/grid/cfgtoollogs/opatch/opatch2012-01-27_18-23-48PM.log
 
 OPatch failed with error code 73

到此为止，我放弃了在虚拟机中安装PSU 11.2.0.2.5（如果要增加虚拟机中的文件系统剩余空间是非常麻烦的事情），但是我认为解决了磁盘空间问题之后，后面应该不会再有太多问题了。另外，如果在真实环境中这些问题可能都不存在，因为真实环境中文件系统的剩余空间应该远远不止3G，也应该不会有Shared Folder权限的问题，不过目录位置的问题应该还是会遇到，希望这里遇到的问题对将要在产品环境中Apply 11.2.0.2.5 PSU的朋友有帮助。

如果你成功Apply了该版本的Patch，那么也可以留言告诉我你遇到了什么障碍。

Share/Save

18:53 MoreSQL: No More NoSQL (2188 Bytes) » myNoSQL

MoreSQL: No More NoSQL:

We at MoreSQL believe in the following axioms:

Universal Applicability: there is no such thing as a problem which cannot be solved with relational databases. It doesn’t matter what you’re storing or how you need to use it. Tabular structures (which may or may not be linked via foreign keys) are the only way to go. End of discussion.

Ends Justify Means: as corollary to axiom 1, we will do whatever it takes to make SQL work for us. Views, stored procedures, cross-database calls: you name it, we’ll do it. Oh and by the way, using ORMs does not mean that you’re trying to shove a round peg into a square hole. They are beautiful and enchanting, OK?

Scale, shmale: relational databases can scale well enough. I mean, Facebook is running on MySQL, for crying out loud! Are you better than Facebook and its 10 trillion active users? I didn’t think so.

I’ve already tattooed myself with MoreSQL and I’m distributing printed leaflets with the axioms in all major squares in town.

Original title and link: MoreSQL: No More NoSQL (NoSQL database©myNoSQL)

18:45 NoSQL Books: Riak Handbook and the Little Redis Book (2136 Bytes) » myNoSQL

A couple of recent books that I’ll be adding to the list of NoSQL books:

Mathias Meyer’s Riak Handbook. You can get an idea of the book by checking Consistent Hashing Explained: The What and the Why, the free sample chapter, and the table of contents.
Karl Seguin’s The Little Redis Book. Karl is at the second free NoSQL book after the The Little MongoDB Book.

Original title and link: NoSQL Books: Riak Handbook and the Little Redis Book (NoSQL database©myNoSQL)

18:23 Measuring User Retention With Hadoop and Hive (1855 Bytes) » myNoSQL

Measuring User Retention With Hadoop and Hive:

A very practical example of how Hive and Hadoop could deliver value when applied to clickstreams, the most common data for each web property:

Hadoop, Hive, and related technologies are formidable tools for unlocking value from data. […] Retention measurements are particularly significant because they paint a detailed picture about the overall stickiness of a product across the entire userbase.

The same clickstream data can be used to calculate visitors’ conversion with the Bayesian discriminant using Hadoop.

Original title and link: Measuring User Retention With Hadoop and Hive (NoSQL database©myNoSQL)

14:00 Log Buffer #256, A Carnival of the Vanities for DBAs (454 Bytes) » The Pythian Blog

Searching for the blogging inspiration? On the look-out for that Eureka moment for your next blockbuster blog post? Finding the exact ingredients for your dream rambling? Well in the Log Buffer Edition, there are some awe-inspiring posts in this Log Buffer #256. Get Inspired, keep blogging. Oracle: If there is a notable technical database conference [...]

07:28 RIP Flash-based My Oracle Support (434 Bytes) » The Pythian Blog

The end is nigh for Adobe Flash at My Oracle Support. The first step will happen during this weekend’s planned maintenance of My Oracle support, when all of MOS will be down for 5 hours starting at midnight eastern, Saturday January 28. Once it comes back up, the unadvertised non-Flash supporthtml.oracle.com will come up as [...]

05:14 The History of NoSQL: This Was Not Our Technology Vendors' Fault (2565 Bytes) » myNoSQL

Werner Vogels in the post about Amazon DynamoDB:

We had been pushing the scalability of commercially available technologies to their limits and finally reached a point where these third party technologies could no longer be used without significant risk. This was not our technology vendors’ fault; Amazon’s scaling needs were beyond the specs for their technologies and we were using them in ways that most of their customers were not. A number of outages at the height of the 2004 holiday shopping season can be traced back to scaling commercial technologies beyond their boundaries.

Here is what I wrote about the history behind NoSQL databases:

Providing decent solutions, up to a point, to a wide range of problems and covering more scenarios than alternative storage solutions existing at that time, made relational databases the de facto storage for the last 30 years. But during the last years, more and more problems crossed the boundaries of what could have been considered decent solutions leading to the need for specialized, better than good enough alternative solutions. And thus NoSQL databases.

It feels rewarding to get such confirmation from people that are at the forefront of NoSQL.