Location: Remote/Hybrid
Job Type: Full-time
About Us: Nichefire is an early stage tech startup specializing in cultural analytics and trend prediction. We help businesses identify and anticipate emerging trends by analyzing vast amounts of text-based data from social media, news, and other digital sources.
Our technology leverages Natural Language Processing (NLP) and Large Language Models (LLMs) to uncover shifts in consumer behavior, industry movements, and cultural trends—before they go mainstream. We build scalable, cloud-based data pipelines to process and transform unstructured data into actionable insights that help brands, agencies, and organizations stay ahead in an evolving market.
As a company, we thrive on innovation, agility, and collaboration, working at the intersection of data science and market intelligence to help businesses make informed decisions about the future.
Considerations: We are interested in every qualified candidate who is eligible to work in the United States. However, we are not able to sponsor visas at this time.
Job Description:
We are seeking an experienced Senior Data Engineer with a strong background in building and managing data collection, processing, and modeling pipelines in the cloud. The ideal candidate will have extensive experience with Airflow, Python, Google Cloud Platform (GCP), Git, and database management. You will be responsible for designing, developing, and maintaining data pipelines that support our NLP and LLM models, ensuring data quality, scalability, and reliability.
Key Responsibilities:
- Design and Develop Data Pipelines: Create, manage, and optimize data collection and processing pipelines using Airflow and GCP to handle large volumes of text-based social media data.
- Cloud Infrastructure Management: Implement and maintain cloud infrastructure on GCP, ensuring high availability, scalability, and security of data processing environments.
- Data Integration: Develop robust data integration solutions to aggregate data from various social media platforms and other sources, ensuring data consistency and reliability.
- NLP and LLM Model Support: Work closely with data scientists and machine learning engineers to support the deployment and maintenance of NLP and LLM models in production.
- Database Management: Design, manage, and optimize databases for storage and retrieval of large-scale text data, ensuring efficient data access and query performance.
- Version Control: Utilize Git for version control and collaboration on codebases, ensuring best practices in code management and deployment.
- Performance Tuning: Monitor and improve the performance of data pipelines, identifying and resolving bottlenecks and inefficiencies.
- Documentation: Maintain comprehensive documentation for all data engineering processes, ensuring transparency and knowledge sharing within the team.
- Collaboration: Work collaboratively with cross-functional teams, including data scientists, product managers, and other stakeholders, to understand data requirements and deliver solutions that meet business needs.
Qualifications:
- Experience: Minimum of 1-3 years of hands-on experience in a data engineering role.
- Technical Expertise: 1-3+ years of experience is a must in designing and implementing ETL/ELT pipelines using Airflow and GCP services (Big Query, Cloud Storage, Pub/Sub, etc.)
- Startup Experience: Experience working at a tech startup is preferred.
- Programming Skills: Advanced knowledge of Python for data processing, automation, and integration tasks.
- Database Skills: Proficiency in SQL and experience with relational and NoSQL databases for handling large-scale data.
- Cloud Knowledge: In-depth understanding of cloud infrastructure, specifically in GCP, including cost management, security best practices, and scalability strategies.
- Version Control: Strong experience with Git for version control, branching strategies, and collaborative development.
- Analytical Skills: Excellent problem-solving and analytical skills, with the ability to identify and resolve complex data engineering challenges.
- Communication: Strong verbal and written communication skills, with the ability to articulate technical concepts to non-technical stakeholders.
- Team Player: Experience working on a small team with a collaborative mindset, supporting colleagues in achieving shared goals.
Preferred Qualifications:
- NLP/LLM Experience: Experience working with NLP and LLM models, particularly in processing and analyzing text-based data.
- Social Media Data: Familiarity with social media data collection and analysis, including APIs and data extraction techniques.
- Certifications: Relevant certifications in GCP or data engineering are a plus.
Benefits:
- Competitive salary and performance-based bonuses.
- Unlimited PTO
- Healthcare stipend offered
- Flexible working hours and remote work options.
- Professional development opportunities and support for certifications.
- Collaborative and innovative work environment.
- Opportunities to work on cutting-edge technologies and challenging projects.
Equal Opportunity Employer:
We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status, or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need.
