Tandon Akash (inter alios) - Advanced Analytics With PySpark. Patterns For Learning From Data At Scale Using Python And Spark.pdf

(9942 KB) Pobierz
Advanced
Analytics with
PySpark
Patterns for Learning from Data at Scale
Using Python and Spark
Akash Tandon,
Sandy Ryza, Uri Laserson,
Sean Owen & Josh Wills
Advanced Analytics with PySpark
The amount of data being generated today is staggering—
and growing. Apache Spark has emerged as the de facto tool
for analyzing big data and is now a critical part of the data
science toolbox. Updated for Spark 3.0, this practical guide
brings together Spark, statistical methods, and real-world
datasets to teach you how to approach analytics problems
using PySpark, Spark’s Python API, and other best practices
in Spark programming.
Data scientists Akash Tandon, Sandy Ryza, Uri Laserson,
Sean Owen, and Josh Wills offer an introduction to the Spark
ecosystem, then dive into patterns that apply common
techniques—including classification, clustering, collaborative
filtering, and anomaly detection—to fields such as genomics,
security, and finance. This updated edition also covers image
processing and the Spark NLP library.
If you have a basic understanding of machine learning and
statistics and you program in Python, this book will get you
started with large-scale data analysis.
Akash Tandon
is cofounder and CTO of
Looppanel. Previously, he worked as a
senior data engineer at Atlan.
Sandy Ryza
leads development of the
Dagster project and is a committer on
Apache Spark.
Uri Laserson
is founder and CTO of
Patch Biosciences. Previously, he
worked on big data and genomics at
Cloudera.
Sean Owen,
a principal solutions
architect focusing on machine learning
and data science at Databricks, is an
Apache Spark committer and PMC
member.
Josh Wills
is a software engineer at
WeaveGrid and the former head of data
engineering at Slack.
•
Familiarize yourself with Spark’s programming model and
ecosystem
•
Learn general approaches in data science
•
Examine complete implementations that analyze large
public datasets
•
Discover which machine learning tools make sense for
particular problems
•
Explore code that can be adapted to many uses
DATA
US $59.99
CAN $74.99
ISBN: 978-1-098-10365-1
Twitter: @oreillymedia
linkedin.com/company/oreilly-media
youtube.com/oreillymedia
55999
9
781098 103651
Advanced Analytics with PySpark
Patterns for Learning from Data at Scale
Using Python and Spark
Akash Tandon, Sandy Ryza, Uri Laserson,
Sean Owen, and Josh Wills
Beijing
Boston Farnham Sebastopol
Tokyo
Advanced Analytics with PySpark
by Akash Tandon, Sandy Ryza, Uri Laserson, Sean Owen, and Josh Wills
Copyright © 2022 Akash Tandon. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are
also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional
sales department: 800-998-9938 or
corporate@oreilly.com.
Acquisitions Editor:
Jessica Haberman
Development Editor:
Jeff Bleiel
Production Editor:
Christopher Faucher
Copyeditor:
Penelope Perkins
Proofreader:
Kim Wimpsett
June 2022:
First Edition
Indexer:
Sue Klefstad
Interior Designer:
David Futato
Cover Designer:
Karen Montgomery
Illustrator:
Kate Dullea
Revision History for the First Edition
2022-06-14:
First Release
See
http://oreilly.com/catalog/errata.csp?isbn=9781098103651
for release details.
The O’Reilly logo is a registered trademark of O’Reilly Media, Inc.
Advanced Analytics with PySpark,
the
cover image, and related trade dress are trademarks of O’Reilly Media, Inc.
The views expressed in this work are those of the authors, and do not represent the publisher’s views.
While the publisher and the authors have used good faith efforts to ensure that the information and
instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility
for errors or omissions, including without limitation responsibility for damages resulting from the use
of or reliance on this work. Use of the information and instructions contained in this work is at your
own risk. If any code samples or other technology this work contains or describes is subject to open
source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use
thereof complies with such licenses and/or rights.
978-1-098-10365-1
[LSI]
Table of Contents
Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
1.
Analyzing Big Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Working with Big Data
Introducing Apache Spark and PySpark
Components
PySpark
Ecosystem
Spark 3.0
PySpark Addresses Challenges of Data Science
Where to Go from Here
Spark Architecture
Installing PySpark
Setting Up Our Data
Analyzing Data with the DataFrame API
Fast Summary Statistics for DataFrames
Pivoting and Reshaping DataFrames
Joining DataFrames and Selecting Features
Scoring and Model Evaluation
Where to Go from Here
2
4
4
6
7
8
8
9
2.
Introduction to Data Analysis with PySpark. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
13
14
17
22
26
28
30
32
34
36
38
40
3.
Recommending Music and the Audioscrobbler Dataset. . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Setting Up the Data
Our Requirements for a Recommender System
Alternating Least Squares Algorithm
iii

Plik z chomika:

Inne pliki z tego folderu:

Inne foldery tego chomika:

Zgłoś jeśli naruszono regulamin