Hello
I have a Senior Big Data Engineer Please find below details and let me know if you have any contract positions.
Name: Devaraj
Visa: GC
Open to relocate
Summary:
v Over 10+ Years of strong experience as Data Engineer including Requirements Analysis, Design Specification and Testing as per Cycle in both Waterfall and Agile methodologies.
v Strong experience in using major components of Hadoop ecosystem components like HDFS, YARN, MapReduce, Hive, Impala, Pig, Sqoop, HBase, Spark, Spark SQL, Kafka, Spark Streaming, Flume, Oozie, Zookeeper, Hue.
v Strong Experience with Amazon Web Services (AWS) Cloud Platform which includes services like EC2, S3, VPC, ELB, IAM, DynamoDB, Cloud Front, Cloud Watch, Route 53, Elastic Beanstalk (EBS), Auto Scaling, Security Groups.
v Good working experience on Spark (spark streaming, spark SQL) with Scala and Kafka. Worked on reading multiple data formats on HDFS using Scala.
v Good understanding and knowledge of NoSQL databases like MongoDB, PostgreSQL, HBase and Cassandra.
v Worked with various formats of files like delimited text files, click stream log files, Apache log files, Avro files, JSON files, XML Files. Mastered in using different columnar file formats like RC, ORC and Parquet formats. Has good understanding of various compression techniques used in Hadoop processing like G-zip, Snappy, LZO etc.
v Experience in Microsoft Azure/Cloud Services like SQL Data Warehouse, Azure SQL Server, Azure Databricks, Azure Data Lake, Azure Blob Storage, Azure Data Factory.
v Hands on experience in writing Map Reduce programs using Java to handle different data sets using Map and Reduce tasks.
v Extensive knowledge in writing Hadoop jobs for data analysis as per the business requirements using Hive and worked on HiveQL queries for required data extraction, join operations, writing custom UDF's as required and having good experience in optimizing Hive Queries.
v Experience in importing and exporting data using Sqoop from f3 to Relational Database systems and vice-versa and load into Hive tables, which are partitioned.
v Developed custom Kafka producer and consumer for different publishing and subscribing to Kafka topics.
v Involved in converting Hive/SQL queries into Spark transformations using Spark Data frames and Scala.
v Experience in using Kafka and Kafka brokers to initiate spark context and processing livestreaming.
v Knowledge of ETL methods for data extraction, transformation and loading in corporate-wide ETL Solutions and Data Warehouse tools for reporting and data analysis.
v Worked with Databricks and Snowflake environments,
v Hands on experience with data ingestion tools Kafka, Flume, and workflow management tools Oozie.
v Used Spark Data Frames API over Cloudera platform to perform analytics on Hive data and Used Spark Data Frame Operations to perform required Validations in the data.
v Experience in extracting files from MongoDB through Sqoop and placed in HDFS and processed.
v Extensive experience working on various databases and database script development using SQL and PL/SQL.
v Worked on various programming languages using IDEs like Eclipse, NetBeans, and IntelliJ, Putty, GIT.
v Have very strong interpersonal skills and the ability to work independently and with the group, can learn quickly and easily adaptable to the working environment.
Technical Skills:
Hadoop Eco System
Hadoop, MapReduce, Spark, HDFS, Sqoop, YARN, Oozie, Hive, Impala, Apache Airflow, HBase.
Programming Languages
Java, PL/SQL, SQL, Python, Scala, PySpark, C, C++, Go
Cluster Mgmt.& monitoring
CDH 4, CDH 5, Horton Works Ambari 2.5
Data Bases
Snowflake, MySQL, SQL Server, Oracle.
NoSQL Data Bases
DynamoDB, Cassandra, HBase
Workflow mgmt. tools
Oozie, Apache Airflow
Visualization & ETL tools
Tableau, Informatica, Talend, PowerBI.
Cloud Technologies
Azure, AWS
IDE’s
Eclipse, Jupyter notebook, Spyder, PyCharm, IntelliJ
Version Control Systems
Git, SVN
Educational Background:
Bachelor of Technology in Electronics and Communication Engineering from SRM University, Chennai. 2009 - 2013.
Thanks & Regards,
Ashok | Sr. Sales Recruiter |
TechSmart Global INC | Ph : 732-791-2779
666 Plainsboro Rd, Suite 1116, Plainsboro, NJ 08536.
