Hadoop程式設計資源

Hadoop 程式設計資源

 

 

content-based image retrieval

 

文仲學長之前在 Data Mining 專題時,對於所收集的資料,其中包含原始程式碼,網站請參考[1]的網址。
另外,其中有一篇是哥倫比亞大學 DVMM Lab 所提供 image dataset 的論文,在他們的網站中[2]有其他關於 content-based image retrieval 的資訊。 (相關資料置於E-Campus)
—————————-
[1] http://thomas.deselaers.de/fire/
[2] http://www.ee.columbia.edu/ln/dvmm/newHome.htm
再次感謝文仲學長提供的資訊!

 

 

 

 

透過Hadoop Map/Reduce,將資料送到MySQL

 這裡對於MySQL寫入 for Hadoop,有不錯的Sample Code:
http://allen501pc.blogspot.com/2011/06/java-hadoop-mapreducemysql.html

 

 

 

Hadoop with Netbeans 函式包裹至單一專案JAR

 感謝潘人豪學長提供:

http://bp.51donate.com/2011/06/hadoop-with-netbeans-jar.html

 

 

 

Install ZooKeeper 3.2

 http://sites.google.com/site/waue0920/Home/zookeeper/zookerper-an-zhuang-cong-ji

 

 

 

HBase 進階課程:範例程式練習

 http://trac.nchc.org.tw/cloud/wiki/NCHCCloudCourse100929_4_HBEX

 

 

 

hadoop + Hbase + thrift + php 安裝設定與程式設計

 Click Here To Visit

 

 

 

HBase 0.20.6 API

 Click Here To Visit

 

 

 

HBase 0.20.x Programming

 Click Here To Visit

 

 

 

Install HBase 0.90.0 base on Hadoop 0.20.2 use tarball

 http://lagunawang.pixnet.net/blog/post/29039445

 

 

 

Pig Setup & Training

 Pig Setup
Pig Training

 

 

 

Setup HBase 0.20

 There is an useful tutorial website:

http://sites.google.com/site/waue0920/Home/hbase/hbase-cong-ji-an-zhuang

 

 

 

wiki xml file 可下載囉

 http://dmlab.cs.nchu.edu.tw/~dmfinalproject/download/freebase-wex-2011-04-16.tar

點選該網址 即可下載(已失效)

 

 

 

Setup Hadoop Distributed Mode

 Setup Hadoop Distributed Mode

 

 

 

 

Mokito framework

 

Mokito is a framework for testing Java applications. It simulates the behavior of complex objects, it facilitates the development of unit testing and error detection.

Mockito is OpenSource and is available at the following address:
http://mockito.org/

Mockito based on the principle of a mock object that simulates the behavior of another object.
The implementation of Mockito testing is done in several steps:

  •  Creation of the mock object.
  •  Description of expected behavior.
  • Using mocks.
  • Checking that the interaction with mocks is correct.

More detail in http://en.kioskea.net/faq/5588-mokito-framework

 

 

 

JUnit

自Chapter 4以後,Hadoop課本會用到的assertThat是來自於JUnit。
這裡有個JUnit的教學資源:
http://caterpillar.onlyfun.net/Gossip/JUnit/index.html

 

 

 

Hadoop with Netbeans 雲端運算開發環境

 感謝 潘人豪學長的教學提供。
http://bp.51donate.com/2011/03/hadoop-netbeans.html

 

 

 

Hadoop: Safe Mode

 

在開啟hadoop ,load namenode的時候,系統會先停在safe mode,使得data node在回報其資料完整性之前,data blocks不會提早做複製的動作。系統若在safe mode,則所有的blocks都是唯讀 (read only)屬性,不能做任何寫入或是修改。直到data node都做出正常回報之後,才會離開safe mode,進行data blocks的複製行為。
若hadoop因某些意外不正常啟動或是關閉,則系統會鎖在safe mode。使得你無法對HDFS做任何變更的動作。因此,如果你要離開safe mode,可下達以下指令:

hadoop dfsadmin -safemode leave

延伸閱讀:
hadoop safe mode的相關操作:
Command:hadoop dfsadmin -safemode [option]
[option]:
enter: 進入safe mode。
get: 顯示系統是否進入safe mode。
wait: 一直等到safe mode結束。
leave: 離開safe mode。

文章來源:
Hadoop Safe Mode

 

 

 

 

Makefile 撰寫教學

 

Makefile撰寫

 

 

 

 

Hadoop setup and map/reduce programming

 

雲端運算基礎課程 (Hadoop簡介、安裝與範例實作)

Hadoop single setup

Hadoop map/reduce programming
注意!它的javadoc解說部分有些錯誤!下載後,解壓縮到/usr/lib/jvm/sun-java6-jdk底下

 

Leave a Reply

你的電子郵件位址並不會被公開。 必要欄位標記為 *