Apache Giraph
Analyzing and delivering personalized information from large-scale graph data using Apache Giraph on secure Hadoop platforms.
Apache Giraph is a large-scale graph processing framework that runs on Apache Hadoop, designed for analyzing massive graphs like web and social networks, reaching trillions of pages and hundreds of millions of users. It follows the bulk-synchronous parallel model, allows checkpoints for automatic restarts, and requires Java 1.8, Maven 3, and specified Hadoop versions for development. Giraph supports various Hadoop releases, including secure Apache Hadoop versions and unsecured Facebook releases, with primary focus on secure versions. It can be tested locally using 'mvn clean test -Dprop.mapred.job.tracker=localhost:9001', and development requires specified Hadoop configurations.
About Apache Giraph
Apache Giraph is a large-scale graph processing framework that runs on Apache Hadoop. It processes and analyzes web and online social graphs, which have grown significantly in size and scale over the past decade, reaching an estimated trillion web pages and hundreds of millions of users in social networking and email sites.
Giraph is designed to play a crucial role in providing relevant and personalized information for users, such as search engine results or news on online social networking sites. It follows the bulk-synchronous parallel model, enabling vertices to send messages during a given superstep. Checkpoints are initiated at user-defined intervals and used for automatic application restarts when workers fail.
To build and test Giraph, you'll need Java 1.8 and Maven 3 or higher, along with the specified Hadoop versions. You can compile, package, and test using various Maven commands with different Hadoop configurations, such as secure versions (Apache Hadoop 1 or 2) or unsecured versions like Facebook Hadoop releases.
After preparing your local filesystem and starting the Hadoop instance, you can run Giraph's unittests on the local Hadoop instance by executing 'mvn clean test -Dprop.mapred.job.tracker=localhost:9001'. For more details on preparing the environment, check the provided instructions in the text.
Giraph supports different versions of Hadoop, including secure versions (Apache Hadoop 1 and 2) and unsecured versions like Facebook Hadoop releases. While it provides limited support for unsecured and Facebook versions with maven profiles 'hadoop_non_secure' and 'hadoop_facebook', respectively, its primary focus is on secure Hadoop releases.