All Things Engineering
Engineering & Developer Experience from the best in class. Culture, Architecture, DevOps and stuff at scale.
🏢 %company% Engineering
AirBnb Engineering
Last posted 2 weeks, 5 days ago
Improving Performance with HTTP Streaming
Improving Performance with HTTP Streaming Improving Performance with HTTP Streaming

How HTTP Streaming can improve page performance and how Airbnb enabled it on an existing codebaseBy: Victor LinIntroductionYou may have heard a joke that the Internet is a series of tubes. In this blog post, we’re going to talk about how we get a cool, refreshing stream of Airbnb.com bytes into your browser as quickly as possible using HTTP Streaming.Let’s first understand what streaming means. Imagine we had a spigot and two options:Fill a big cup, and then pour it all down the tube (the “buffered” strategy)Connect the spigot directly to the tube (the “streaming” strategy)In the buffered strategy, everything happens sequentially — our servers first generate the entire response into a buffe…

2 weeks, 5 days ago @ medium.com
Journey Platform: A low-code tool for creating interactive user workflows
Journey Platform: A low-code tool for creating interactive user workflows Journey Platform: A low-code tool for creating interactive user workflows

Journey Platform: Low-code notification workflow platform that allows technical and non-technical users to create complex workflows through a simple drag and drop user interface.By: Arjun Raman, Ken Snyder, Mengting LiIntroductionEffective communication hinges on delivering the right message, to the right audience, at the right time. At Airbnb, our goal is to engage our users — both guests and hosts — by delivering inspirational and informational notifications through various channels, such as email or in-app messages.Historically at Airbnb, complex notification workflows have been solely managed by engineering teams, with each workflow requiring the deployment of code. As our platform evol…

3 weeks, 4 days ago @ medium.com
Flexible Continuous Integration for iOS
Flexible Continuous Integration for iOS Flexible Continuous Integration for iOS

How Airbnb leverages AWS, Packer, and Terraform to update macOS on hundreds Cl machines in hours instead of daysBy: Michael Bachand, Xianwen ChenAt Airbnb, we run a comprehensive suite of continuous integration (CI) jobs before each iOS code change is merged. These jobs ensure that the main branch remains stable by executing critical developer workflows like building the iOS application and running tests. We also schedule jobs that perform periodic tasks like reporting metrics and uploading artifacts.Many of our iOS CI jobs execute on Macs, which enables running developer tools provided by Apple. CI jobs for all other platforms at Airbnb execute in containers on Amazon EC2 Linux instances. …

3 weeks, 5 days ago @ medium.com
Netflix Engineering Netflix Engineering
Last posted 6 hours ago
Native Frame Rate Playback
Native Frame Rate Playback Native Frame Rate Playback

by Akshay Garg, Roger QueroIntroductionMaximizing immersion for our members is an important goal for the Netflix product and engineering teams to keep our members entertained and fully engaged in our content. Leveraging a good mix of mature and cutting-edge client device technologies to deliver a smooth playback experience with glitch-free in-app transitions is an important step towards achieving this goal. In this article we explain our journey towards productizing a better viewing experience for our members by utilizing features and capabilities in consumer streaming devices.If you have a streaming device connected to your TV, such as a Roku Set Top Box (STB) or an Amazon FireTV Stick, yo…

6 hours ago @ netflixtechblog.com
Ensuring the Successful Launch of Ads on Netflix
Ensuring the Successful Launch of Ads on Netflix Ensuring the Successful Launch of Ads on Netflix

By Jose Fernandez, Ed Barker, Hank JacobsIntroductionIn November 2022, we introduced a brand new tier — Basic with ads. This tier extended existing infrastructure by adding new backend components and a new remote call to our ads partner on the playback path. As we were gearing up for launch, we wanted to ensure it would go as smoothly as possible. To do this, we devised a novel way to simulate the projected traffic weeks ahead of launch by building upon the traffic migration framework described here. We used this simulation to help us surface problems of scale and validate our Ads algorithms.Basic with ads was launched worldwide on November 3rd. In this blog post, we’ll discuss the methods …

4 days, 3 hours ago @ netflixtechblog.com
Debugging a FUSE deadlock in the Linux kernel
Debugging a FUSE deadlock in the Linux kernel Debugging a FUSE deadlock in the Linux kernel

Tycho AndersenI work on the Compute team at Netflix, which is charged with managing all AWS and containerized workloads at Netflix, including autoscaling, deployment of containers, issue remediation, etc.This particular issue involved a custom internal FUSE filesystem: ndrive. It had been festering for some time, but needed someone to sit down and look at it in anger. This blog post describes how I poked at /procto get a sense of what was going on, before posting the issue to the kernel mailing list and getting schooled on how the kernel’s wait code actually works!Symptom: Stuck Docker Kill & A Zombie ProcessWe had a stuck docker API call:goroutine 146 [select, 8817 minutes]:net/http.(*pers…

2 weeks, 3 days ago @ netflixtechblog.com
ABAC on SpiceDB: Enabling Netflix’s Complex Identity Types
ABAC on SpiceDB: Enabling Netflix’s Complex Identity Types ABAC on SpiceDB: Enabling Netflix’s Complex Identity Types

By Chris Wolfe, Joey Schorr, and Victor Roldán BetancortIntroductionThe authorization team at Netflix recently sponsored work to add Attribute Based Access Control (ABAC) support to AuthZed’s open source Google Zanzibar inspired authorization system, SpiceDB. Netflix required attribute support in SpiceDB to support core Netflix application identity constructs. This post discusses why Netflix wanted ABAC support in SpiceDB, how Netflix collaborated with AuthZed, the end result–SpiceDB Caveats, and how Netflix may leverage this new feature.Netflix is always looking for security, ergonomic, or efficiency improvements, and this extends to authorization tools. Google Zanzibar is exciting to Netf…

2 weeks, 3 days ago @ netflixtechblog.com
Migrating Critical Traffic At Scale with No Downtime — Part 1
Migrating Critical Traffic At Scale with No Downtime — Part 1 Migrating Critical Traffic At Scale with No Downtime — Part 1

Migrating Critical Traffic At Scale with No Downtime — Part 1Shyam Gala, Javier Fernandez-Ivern, Anup Rokkam Pratap, Devang ShahHundreds of millions of customers tune into Netflix every day, expecting an uninterrupted and immersive streaming experience. Behind the scenes, a myriad of systems and services are involved in orchestrating the product experience. These backend systems are consistently being evolved and optimized to meet and exceed customer and product expectations.When undertaking system migrations, one of the main challenges is establishing confidence and seamlessly transitioning the traffic to the upgraded architecture without adversely impacting the customer experience. This b…

1 month ago @ netflixtechblog.com
Improved Alerting with Atlas Streaming Eval
Improved Alerting with Atlas Streaming Eval Improved Alerting with Atlas Streaming Eval

Ruchir Jha, Brian Harrington, Yingwu ZhaoTL;DRStreaming alert evaluation scales much better than the traditional approach of polling time-series databases.It allows us to overcome high dimensionality/cardinality limitations of the time-series database.It opens doors to support more exciting use-cases.Engineers want their alerting system to be realtime, reliable, and actionable. While actionability is subjective and may vary by use-case, reliability is non-negotiable. In other words, false positives are bad but false negatives are the absolute worst!A few years ago, we were paged by our SRE team due to our Metrics Alerting System falling behind — critical application health alerts reached en…

1 month, 1 week ago @ netflixtechblog.com
Building a Media Understanding Platform for ML Innovations
Building a Media Understanding Platform for ML Innovations Building a Media Understanding Platform for ML Innovations

By Guru Tahasildar, Amir Ziai, Jonathan Solórzano-Hamilton, Kelli Griggs, Vi IyengarIntroductionNetflix leverages machine learning to create the best media for our members. Earlier we shared the details of one of these algorithms, introduced how our platform team is evolving the media-specific machine learning ecosystem, and discussed how data from these algorithms gets stored in our annotation service.Much of the ML literature focuses on model training, evaluation, and scoring. In this post, we will explore an understudied aspect of the ML lifecycle: integration of model outputs into applications.An example of using Machine Learning to find shots of Eleven in Stranger Things and surfacing …

2 months, 3 weeks ago @ netflixtechblog.com
Elasticsearch Indexing Strategy in Asset Management Platform (AMP)
Elasticsearch Indexing Strategy in Asset Management Platform (AMP) Elasticsearch Indexing Strategy in Asset Management Platform (AMP)

By Burak Bacioglu, Meenakshi JindalAsset Management at NetflixAt Netflix, all of our digital media assets (images, videos, text, etc.) are stored in secure storage layers. We built an asset management platform (AMP), codenamed Amsterdam, in order to easily organize and manage the metadata, schema, relations and permissions of these assets. It is also responsible for asset discovery, validation, sharing, and for triggering workflows.Amsterdam service utilizes various solutions such as Cassandra, Kafka, Zookeeper, EvCache etc. In this blog, we will be focusing on how we utilize Elasticsearch for indexing and search the assets.Amsterdam is built on top of three storage layers.The first layer, …

2 months, 3 weeks ago @ netflixtechblog.com
Data Reprocessing Pipeline in Asset Management Platform @Netflix
Data Reprocessing Pipeline in Asset Management Platform @Netflix Data Reprocessing Pipeline in Asset Management Platform @Netflix

By Meenakshi JindalOverviewAt Netflix, we built the asset management platform (AMP) as a centralized service to organize, store and discover the digital media assets created during the movie production. Studio applications use this service to store their media assets, which then goes through an asset cycle of schema validation, versioning, access control, sharing, triggering configured workflows like inspection, proxy generation etc. This platform has evolved from supporting studio applications to data science applications, machine-learning applications to discover the assets metadata, and build various data facts.During this evolution, quite often we receive requests to update the existing…

2 months, 3 weeks ago @ netflixtechblog.com
Data ingestion pipeline with Operation Management
Data ingestion pipeline with Operation Management Data ingestion pipeline with Operation Management

by Varun Sekhri, Meenakshi Jindal, Burak BaciogluIntroductionAt Netflix, to promote and recommend the content to users in the best possible way there are many Media Algorithm teams which work hand in hand with content creators and editors. Several of these algorithms aim to improve different manual workflows so that we show the personalized promotional image, trailer or the show to the user.These media focused machine learning algorithms as well as other teams generate a lot of data from the media files, which we described in our previous blog, are stored as annotations in Marken. We designed a unique concept called Annotation Operations which allows teams to create data pipelines and easil…

3 months ago @ netflixtechblog.com
Scaling Media Machine Learning at Netflix
Scaling Media Machine Learning at Netflix Scaling Media Machine Learning at Netflix

By Gustavo Carmo, Elliot Chow, Nagendra Kamath, Akshay Modi, Jason Ge, Wenbing Bai, Jackson de Campos, Lingyi Liu, Pablo Delgado, Meenakshi Jindal, Boris Chen, Vi Iyengar, Kelli Griggs, Amir Ziai, Prasanna Padmanabhan, and Hossein TaghaviIntroductionIn 2007, Netflix started offering streaming alongside its DVD shipping services. As the catalog grew and users adopted streaming, so did the opportunities for creating and improving our recommendations. With a catalog spanning thousands of shows and a diverse member base spanning millions of accounts, recommending the right show to our members is crucial.Why should members care about any particular show that we recommend? Trailers and artworks p…

3 months, 3 weeks ago @ netflixtechblog.com
Pinterest Engineering
Last posted 1 week, 6 days ago
Pacer: Pinterest’s New Generation of Asynchronous Computing Platform
Pacer: Pinterest’s New Generation of Asynchronous Computing Platform Pacer: Pinterest’s New Generation of Asynchronous Computing Platform

Qi Li | Software Engineer, Core-Services; Zhihuang Chen | Software Engineer, Core-Services; Ping Jin | Engineer manager, Core ServicesAt Pinterest, a wide range of functionalities and features for various business needs and products are supported by an asynchronous job execution platform called Pinlater, which was open-sourced a few years ago. Use cases on the platform span from saving Pins by Pinners, to notifying Pinners about various updates, to processing images/videos etc. Pinlater handles billions of job executions on a daily basis. The platform supports many desirable features, like at-least-once semantics, job scheduling for future execution, and dequeuing/processing speed control o…

1 week, 6 days ago @ medium.com
Warden: Real Time Anomaly Detection at Pinterest
Warden: Real Time Anomaly Detection at Pinterest Warden: Real Time Anomaly Detection at Pinterest

Isabel Tallam | Sw Eng, Real Time Analytics; Charles Wu | Sw Eng, Real Time Analytics; Kapil Bajaj | Eng Manager, Real Time AnalyticsDetecting anomalous events has been becoming increasingly important in recent years at Pinterest. Anomalous events, broadly defined, are rare occurrences that deviate from normal or expected behavior. Because these types of events can be found almost anywhere, opportunities and applications for anomaly detection are vast. At Pinterest, we have explored leveraging anomaly detection, specifically our Warden Anomaly Detection Platform, for several use cases (which we’ll get into in this post). With the positive results we are seeing, we are planning to continue t…

2 weeks, 5 days ago @ medium.com
An ML based approach to proactive advertiser churn prevention
An ML based approach to proactive advertiser churn prevention An ML based approach to proactive advertiser churn prevention

Erika Sun ML Engineer | Advertiser Growth Modeling Team; Ogheneovo Dibie Engineering Manager | Advertiser Growth Modeling TeamPhoto by Jason Blackeye on UnsplashSummaryIn this blog post, we describe a Machine Learning (ML) powered proactive churn prevention solution that was prototyped with our small & medium business (SMB) advertisers. Results from our initial experiment suggest that we can detect future churn with a high degree of predictive power and consequently empower our sales partners in mitigating churn. ML-powered proactive churn prevention can achieve better results than traditional reactive manual effort.IntroductionLike many ads-based businesses, at Pinterest, we are intently f…

3 weeks, 6 days ago @ medium.com
Facebook
Last posted 2 weeks, 4 days ago
MSVP is Meta’s first video processing ASIC
MSVP is Meta’s first video processing ASIC

[...]

Read More...

The post MSVP is Meta’s first video processing ASIC appeared first on Engineering at Meta.

2 weeks, 4 days ago @ ai.facebook.com
Meta introduces its first-generation AI inference accelerator
Meta introduces its first-generation AI inference accelerator

[...]

Read More...

The post Meta introduces its first-generation AI inference accelerator appeared first on Engineering at Meta.

2 weeks, 4 days ago @ ai.facebook.com
Improving Instagram notification management with machine learning and causal inference
Improving Instagram notification management with machine learning and causal inference

We’re sharing how Meta is applying statistics and machine learning (ML) to improve notification personalization and management on Instagram – particularly on daily digest push notifications. By using causal inference and ML to identify highly active users who are likely to see more content organically, we have been able to reduce the number of notifications [...]

Read More...

The post Improving Instagram notification management with machine learning and causal inference appeared first on Engineering at Meta.

7 months, 1 week ago @ engineering.fb.com
Scaling data ingestion for machine learning training at Meta
Scaling data ingestion for machine learning training at Meta

Many of Meta’s products, such as search, ads ranking and Marketplace, utilize AI models to continuously improve user experiences. As the performance of hardware we use to support training infrastructure increases, we need to scale our data ingestion infrastructure accordingly to handle workloads more efficiently. GPUs, which are used for training infrastructure, tend to double [...]

Read More...

The post Scaling data ingestion for machine learning training at Meta appeared first on Engineering at Meta.

8 months, 2 weeks ago @ engineering.fb.com
Uber Engineering
Last posted None
Spotify Engineering Spotify Engineering
Last posted 1 week, 4 days ago
Multiple Layers of Abstraction in Design Systems
Multiple Layers of Abstraction in Design Systems

Check out our previous post — Customization & Configuration in Design Systems — for more about how we define APIs for design systems. The previous post on this topic explored two common paths that emerge in APIs — customization and configuration — each with its own pros and cons. Customization provides freedom and speed, while [...]

The post Multiple Layers of Abstraction in Design Systems appeared first on Spotify Engineering.

1 week, 4 days ago @ engineering.atspotify.com
Fleet Management at Spotify (Part 3): Fleet-wide Refactoring
Fleet Management at Spotify (Part 3): Fleet-wide Refactoring

Check out Part 1 and Part 2 of our series on Fleet Management at Spotify and how we manage our software at scale. For the third part of this Fleet Management series, we’ll discuss what we call “fleet-wide refactoring” of code across thousands of Git repos: the tools we’ve built to enable fleet-wide refactoring and [...]

The post Fleet Management at Spotify (Part 3): Fleet-wide Refactoring appeared first on Spotify Engineering.

3 weeks ago @ engineering.atspotify.com
TV Spatial Navigation
TV Spatial Navigation

Very often, when developing user interfaces, a lot of focus is given to the visuals, as that’s what primarily gets a user’s attention. However, other aspects are equally impactful for making the experience great but can be overlooked, like the user’s input – how the user interacts with the app. It is completely understandable that [...]

The post TV Spatial Navigation appeared first on Spotify Engineering.

3 weeks, 5 days ago @ engineering.atspotify.com
Fleet Management at Spotify (Part 2): The Path to Declarative Infrastructure
Fleet Management at Spotify (Part 2): The Path to Declarative Infrastructure

This is part 2 in our series on Fleet Management at Spotify and how we manage our software at scale. See also part 1 and part 3. At Spotify, we adopted the declarative infrastructure paradigm to evolve our infrastructure platform’s configuration management and control plane approach, allowing us to manage hundreds of thousands of cloud [...]

The post Fleet Management at Spotify (Part 2): The Path to Declarative Infrastructure appeared first on Spotify Engineering.

1 month ago @ engineering.atspotify.com
Humans + Machines: A Look Behind the Playlists Powered by Spotify’s Algotorial Technology
Humans + Machines: A Look Behind the Playlists Powered by Spotify’s Algotorial Technology

TL;DR Since 2017, Spotify has been working to create a better listening experience for our users by creating algorithmically personalized playlists powered by the expertise of our curators. The outcome of these efforts has resulted in the technology we call “Algotorial.” The best of both worlds: Editorial and Algorithms Spotify has a library of playlists [...]

The post Humans + Machines: A Look Behind the Playlists Powered by Spotify’s Algotorial Technology appeared first on Spotify Engineering.

1 month, 1 week ago @ engineering.atspotify.com
SCRIBD
Last posted None
Atlassian Atlassian
Last posted 5 days, 3 hours ago
Get ready for DevDay: The Meetup
Get ready for DevDay: The Meetup Get ready for DevDay: The Meetup

We've condensed the agenda into a free 90-minute webinar packed full of knowledge-sharing, called DevDay: The Meetup on 28 June 2023.

When you register, you'll also access free hands-on learning and technical workshops held all week long, led by Atlassian engineers and developer advocates.

Here's what else you can expect from DevDay: The Meetup:Opening keynote with Keran McKenzie, Head of Atlassian EcosystemThe road ahead is changing.

Jill Patterson, Senior Product Manager, ScriptRunner for Jira Cloud from Adaptavist, another Forge pioneer, who recently released Scriptrunner Behaviours on Forge.

Grab your free pass for DevDay: The Meetup and sign up for free workshops today.

5 days, 3 hours ago @ blog.developer.atlassian.com
Data residency update: realm migration for Connect partner preview
Data residency update: realm migration for Connect partner preview Data residency update: realm migration for Connect partner preview

With the Preview release, we encourage partners with Connect apps to start using the realm migration service and associated APIs to complete the data residency solution for your customers.

Privacy & Security tab App data residency UI on admin.atlassian.comWith data residency information visible on the Marketplace Privacy & Security tab and on admin.atlassian.com, partners who offer data residency can now highlight this feature to customers more easily.

How to get started with realm migration for Connect appsCustomers are currently able to move their Jira or Confluence cloud data to different realms to meet their data residency needs.

Even if an app supports realm pinning in the correct regi…

1 week ago @ blog.developer.atlassian.com
Handling rate limiting in JavaScript
Handling rate limiting in JavaScript Handling rate limiting in JavaScript

Inadequate handling of rate limiting responses often leads to poor user experiences and reliability issues.

Whilst the Jira Cloud rate limiting and Confluence Cloud rate limiting guides both have pseudo code explaining how to handle rate limiting responses, this article provides more concrete guidance for a standard Node.js service.

this.addNoResponseHeaders : this.addRetryAfterResponseHeader } return mockFetchInfo; } else if (randomValue < 0.75) { const mockFetchInfo: MockFetchInfo = { responseStatusCode: 500, responseStatusText: 'Internal Server Error', addResponseHeaders: this.addNoResponseHeaders } return mockFetchInfo; } else { const mockFetchInfo: MockFetchInfo = { responseStatusCode:…

1 week, 3 days ago @ blog.developer.atlassian.com
Forge Roadmap Webinar Recap: Q1 2023
Forge Roadmap Webinar Recap: Q1 2023 Forge Roadmap Webinar Recap: Q1 2023

Forge roadmap updates: What's nextWith these broader changes addressed, Forge product managers dove into individual updates across Forge.

Here's a summary of our progress so far:Implementing Forge UI extensibility in some areas of Bitbucket Cloud is relatively complete, and we've finished work to enable Forge functions and Forge events.

User consent screenWork is still underway to remove the need for all users to consent to Forge apps.

Client-side UI kitClient-side UI kit adds performance and flexibility to the existing UI kit while making it compatible with Custom UI.

Q: The fact that Forge apps are not always deployed in the cloud region closest to the customer causes latency.

1 week, 5 days ago @ blog.developer.atlassian.com
Recap | May 2023 Tech Partner Marketing Hour
Recap | May 2023 Tech Partner Marketing Hour Recap | May 2023 Tech Partner Marketing Hour

Step 1: Define your audience and goalsThere are key considerations when creating marketing assets.

B ) Existing Customers → Customers want a quick, easy and frictionless experience when getting started with a new app.

Atlassian Community EventsAtlassian Community Events are free, user-led meetups happening in cities all over the world.

A: You can sign up to host community events on Atlassian Community Events (ACE) by signing up to be a leader in your city.

The Tech Insider newsletter is delivered straight to your inbox and is focused on best practices for marketing your app.

3 weeks ago @ blog.developer.atlassian.com
Event Recap: App Week ‘23 Berlin
Event Recap: App Week ‘23 Berlin Event Recap: App Week ‘23 Berlin

Reading Time: 5 minutesIn February, we celebrated the return of App Week – the first in-person Atlassian developer event we've hosted in over three years.

Takeaways from App Week '23In addition to coding time, App Week 2023 was action packed, with presentations, research sessions, demos, and roundtable discussions.

Testing new featuresSeveral Early Access Programs (EAPs) kicked off at App Week, allowing attendees to try new Forge features.

Synergy with Atlassian Presents: UnleashThis year, App Week shared a venue with Atlassian Presents: Unleash, Atlassian’s event for Agile and DevOps.

This crossover meant App Week attendees had access to additional sessions with thought leaders and experts…

4 weeks ago @ blog.developer.atlassian.com
Guide to customer conversations about server end of support and cloud migration
Guide to customer conversations about server end of support and cloud migration Guide to customer conversations about server end of support and cloud migration

During this period, we want to support customers and Marketplace Partners in aligning your future roadmap and cloud migration plans.

The Cloud Fortified program is an option for Marketplace Partners to signal you've made additional investments in security, reliability and support for your cloud apps.

Details : We understand it is sometimes difficult for Marketplace Partners with a large customer base to respond to a large volume of customer requests.

Be ready to address common questions about server appsQuestion: Will I still be able to renew my server app now, before Feb 15, 2024?

Answer : Server apps will no longer be approved on Atlassian Marketplace apps after Feb 15, 2024.

2 months ago @ blog.developer.atlassian.com
Privacy & Security tab and app data residency UI: gradual rollouts begin today
Privacy & Security tab and app data residency UI: gradual rollouts begin today Privacy & Security tab and app data residency UI: gradual rollouts begin today

We are also introducing a new UI for app data residency, which will display app data residency options on admin.atlassian.com.

Privacy & Security tab App data residency UI on admin.atlassian.comThese experiences are designed to streamline the app assessment process and help us create a more trustworthy cloud experience for customers with apps.

If you haven't yet, please review our documentation on data residency and the most recent data residency update.

100% of customers with self service data residency will see the app data residency UI and options in admin.atlassian.com.

100% of customers with self service data residency will see the app data residency UI and options in admin.atlassian.c…

2 months, 1 week ago @ blog.developer.atlassian.com
Using Post-Deployment Verification to Ensure Quality in your Marketplace Apps
Using Post-Deployment Verification to Ensure Quality in your Marketplace Apps Using Post-Deployment Verification to Ensure Quality in your Marketplace Apps

We'll be showing screenshots of our app, Advanced Microsoft Teams Jira Connector.

For example, Microsoft Teams Jira Connector global configuration page object which contains an iframe of the Marketplace app configuration page.

For example, Microsoft Teams Jira Connector global configuration page object which contains an iframe of the Marketplace app configuration page.

We didn't want to have test cases duplications between Jira Cloud and Data Center when they can be covered by single independent of the Jira environment test case.

// pdv-environment.js const program = require("commander"); const shell = require("shelljs"); program .version("5.0.0") .option("-a, --app [app]", "Application to …

2 months, 2 weeks ago @ blog.developer.atlassian.com
Find it, fix it, launch it: 5 Tips for debugging Forge apps
Find it, fix it, launch it: 5 Tips for debugging Forge apps Find it, fix it, launch it: 5 Tips for debugging Forge apps

We spoke to Dmitrii Sorin, an Atlassian engineer who works on the team building Forge, to round up 5 useful Forge debugging practices, skills, and tools.

Some developers debug their apps by changing the code and running forge deploy; however, this isn't ideal, as it's much slower than using forge tunnel, and there's a daily deploy limit per Forge environment.

While Forge does support logging via the forge logs CLI command, the best way to view your logs is in the Developer Console.

You run forge tunnel, and Forge starts a tunnel automatically, and the same thing happens: the app on your local machine will start receiving requests.

When it comes to building and debugging Forge apps, the Forg…

3 months ago @ blog.developer.atlassian.com
Infrastructure
AWS
Last posted an hour ago
Introducing in-place version upgrades with Amazon MWAA
Introducing in-place version upgrades with Amazon MWAA

Today, AWS is announcing the availability of in-place version upgrades for Amazon Managed Workflow for Apache Airflow (Amazon MWAA). This enhancement allows you to seamlessly upgrade your existing Apache Airflow version 2.x environments to newer available versions while retaining the workflow run history and environment configurations. You can now take advantage of the latest capabilities […]

an hour ago @ aws.amazon.com
Announcing Amazon Keyspaces Multi-Region Replication
Announcing Amazon Keyspaces Multi-Region Replication

Amazon Keyspaces (for Apache Cassandra) is a scalable, highly available, and managed Apache Cassandra-compatible database service. With Amazon Keyspaces, you can run your Cassandra workloads on AWS using the same Cassandra application code and developer tools that you use today. Today we are introducing Amazon Keyspaces Multi-Region Replication. Amazon Keyspaces Multi-Region Replication is a new […]

an hour ago @ aws.amazon.com
Use Amazon SageMaker Canvas to build machine learning models using Parquet data from Amazon Athena and AWS Lake Formation
Use Amazon SageMaker Canvas to build machine learning models using Parquet data from Amazon Athena and AWS Lake Formation

Data is the foundation for machine learning (ML) algorithms. One of the most common formats for storing large amounts of data is Apache Parquet due to its compact and highly efficient format. This means that business analysts who want to extract insights from the large volumes of data in their data warehouse must frequently use […]

2 hours ago @ aws.amazon.com
Amazon SageMaker Automatic Model Tuning now automatically chooses tuning configurations to improve usability and cost efficiency
Amazon SageMaker Automatic Model Tuning now automatically chooses tuning configurations to improve usability and cost efficiency

Amazon SageMaker Automatic Model Tuning has introduced Autotune, a new feature to automatically choose hyperparameters on your behalf. This provides an accelerated and more efficient way to find hyperparameter ranges, and can provide significant optimized budget and time management for your automatic model tuning jobs. In this post, we discuss this new capability and some […]

2 hours ago @ aws.amazon.com
Train a Large Language Model on a single Amazon SageMaker GPU with Hugging Face and LoRA
Train a Large Language Model on a single Amazon SageMaker GPU with Hugging Face and LoRA

This post is co-written with Philipp Schmid from Hugging Face. We have all heard about the progress being made in the field of large language models (LLMs) and the ever-growing number of problem sets where LLMs are providing valuable insights. Large models, when trained over massive datasets and several tasks, are also able to generalize […]

3 hours ago @ aws.amazon.com
Advanced patterns with AWS SDK for pandas on AWS Glue for Ray
Advanced patterns with AWS SDK for pandas on AWS Glue for Ray

AWS SDK for pandas is a popular Python library among data scientists, data engineers, and developers. It simplifies interaction between AWS data and analytics services and pandas DataFrames. It allows easy integration and data movement between 22 types of data stores, including Amazon Simple Storage Service (Amazon S3), Amazon Athena, Amazon Redshift, and Amazon OpenSearch […]

4 hours ago @ aws.amazon.com
Announcing the launch of new Hugging Face LLM Inference containers on Amazon SageMaker
Announcing the launch of new Hugging Face LLM Inference containers on Amazon SageMaker

This post is co-written with Philipp Schmid and Jeff Boudier from Hugging Face. Today, as part of Amazon Web Services’ partnership with Hugging Face, we are excited to announce the release of a new Hugging Face Deep Learning Container (DLC) for inference with Large Language Models (LLMs). This new Hugging Face LLM DLC is powered […]

4 hours ago @ aws.amazon.com
Enable complex row-level security in embedded dashboards for non-provisioned users in Amazon QuickSight with OR-based tags
Enable complex row-level security in embedded dashboards for non-provisioned users in Amazon QuickSight with OR-based tags

Amazon QuickSight is a fully managed, cloud-native business intelligence (BI) service that makes it easy to connect to your data, create interactive dashboards, and share these with tens of thousands of users, both within QuickSight and embedded in your software as a service (SaaS) applications. QuickSight Enterprise edition started supporting nested conditions within row-level security […]

5 hours ago @ aws.amazon.com
Best practices for migrating SQL Server MERGE statements to Babelfish for Aurora PostgreSQL
Best practices for migrating SQL Server MERGE statements to Babelfish for Aurora PostgreSQL

To migrate a SQL Server database to Babelfish for Aurora PostgreSQL, you usually need to perform both automated and manual tasks. The automated tasks involve automatic code conversion using the Babelfish Compass tool with the -rewrite flag and data migration using AWS Database Migration Service (AWS DMS). The manual tasks involve database compatibility check using […]

9 hours ago @ aws.amazon.com
BWH Hotels scales enterprise business intelligence adoption while reducing costs with Amazon QuickSight
BWH Hotels scales enterprise business intelligence adoption while reducing costs with Amazon QuickSight

This is a guest post by Joseph Landucci from BWH Hotels. In their own words, “BWH Hotels is a leading, global hospitality enterprise comprised of three hotel companies, including WorldHotels, Best Western Hotels & Resorts and SureStay Hotels. Our mission is to deliver trusted guest experiences, drive hotel success and foster a caring, inclusive culture that respects […]

3 days, 3 hours ago @ aws.amazon.com
Stream data with Amazon DocumentDB, Amazon MSK Serverless, and Amazon MSK Connect
Stream data with Amazon DocumentDB, Amazon MSK Serverless, and Amazon MSK Connect

A common trend in modern application development and data processing is the use of Apache Kafka as a standard delivery mechanism for data pipeline and fan-out approach. Amazon Managed Streaming for Apache Kafka (Amazon MSK) is a fully-managed, highly available, and secure service that makes it simple for developers and DevOps managers to run applications […]

3 days, 4 hours ago @ aws.amazon.com
Migrate from Google BigQuery to Amazon Redshift using AWS Glue and Custom Auto Loader Framework
Migrate from Google BigQuery to Amazon Redshift using AWS Glue and Custom Auto Loader Framework

Amazon Redshift is a widely used, fully managed, petabyte-scale cloud data warehouse. Tens of thousands of customers use Amazon Redshift to process exabytes of data every day to power their analytic workloads. Customers are looking for tools that make it easier to migrate from other data warehouses, such as Google BigQuery, to Amazon Redshift to […]

3 days, 7 hours ago @ aws.amazon.com
Cross-account Amazon Aurora MySQL migration with Aurora cloning and binlog replication for reduced downtime
Cross-account Amazon Aurora MySQL migration with Aurora cloning and binlog replication for reduced downtime

Migrating your relational databases on Amazon Aurora MySQL-Compatible Edition from one AWS account to another AWS account is a common use case in scenarios when you acquire, sell, or merge operations, or if you’re reorganizing your AWS accounts and organizational structure. These migrations can be complex, especially for large workloads. It is important to perform […]

3 days, 7 hours ago @ aws.amazon.com
Real-time inference using deep learning within Amazon Kinesis Data Analytics for Apache Flink
Real-time inference using deep learning within Amazon Kinesis Data Analytics for Apache Flink

Apache Flink is a framework and distributed processing engine for stateful computations over data streams. Amazon Kinesis Data Analytics for Apache Flink is a fully managed service that enables you to use an Apache Flink application to process streaming data. The Deep Java Library (DJL) is an open-source, high-level, engine-agnostic Java framework for deep learning. […]

4 days, 2 hours ago @ aws.amazon.com
Migrate SQL Server databases in Azure to Amazon RDS Custom for SQL Server using backup and bacpac files and Amazon S3
Migrate SQL Server databases in Azure to Amazon RDS Custom for SQL Server using backup and bacpac files and Amazon S3

In this post we show you how to migrate from Azure to Amazon Relational Database Service (Amazon RDS) Custom for SQL Server. using the native backup and restore method. We also dive deep into the data-tier application backup package file. Amazon RDS Custom for SQL Server is a managed database service for legacy, custom, and […]

4 days, 4 hours ago @ aws.amazon.com
AWS
Last posted an hour ago
Implement a multi-object tracking solution on a custom dataset with Amazon SageMaker
Implement a multi-object tracking solution on a custom dataset with Amazon SageMaker

The demand for multi-object tracking (MOT) in video analysis has increased significantly in many industries, such as live sports, manufacturing, and traffic monitoring. For example, in live sports, MOT can track soccer players in real time to analyze physical performance such as real-time speed and moving distance. Since its introduction in 2021, ByteTrack remains to […]

4 days, 4 hours ago @ aws.amazon.com
Model molecular SMILES data with Amazon Neptune and RDKit
Model molecular SMILES data with Amazon Neptune and RDKit

Modeling chemical structures can be a complex and tedious process, even with the help of modern programs and technology. The ability to explore chemical structures at the most fundamental level of atoms and the bonds that connect them is an essential process in drug discovery, pharmaceutical research, and chemical engineering. By infusing chemical research with […]

4 days, 4 hours ago @ aws.amazon.com
Build hypothetical indexes in Amazon RDS for PostgreSQL with HypoPG
Build hypothetical indexes in Amazon RDS for PostgreSQL with HypoPG

Indexes in PostgreSQL are essential for improving the performance of database queries. They serve as data structures that organize and optimize the retrieval of information from database tables. By creating indexes on specific columns, PostgreSQL can locate and access relevant data more efficiently. Indexes work by creating a separate data structure that contains a sorted […]

4 days, 4 hours ago @ aws.amazon.com
Create AMIs with AWS CodePipeline
Create AMIs with AWS CodePipeline

One way to alleviate the pain of drift in your Amazon Web Services (AWS) workloads is to use a GitOps workflow to automate Amazon Machine Image (AMI) builds with AWS CodePipeline. In this post, we show you how to deploy a continuous integration and continuous delivery (CI/CD) pipeline, managed by CodePipeline, to build AMIs.

4 days, 7 hours ago @ aws.amazon.com
Version 1 of the AWS Cloud Development Kit (AWS CDK) has reached end-of-support
Version 1 of the AWS Cloud Development Kit (AWS CDK) has reached end-of-support

Since its introduction in 2018, AWS CDK has gained significant traction among developers for building Infrastructure as Code solutions. As technology advances and new features emerge, it is inevitable that older versions of tools must reach their end-of-support. As of June 1, 2023, AWS CDK v1 has officially reached its end-of-support. Consequently, AWS will no […]

4 days, 10 hours ago @ aws.amazon.com
Amazon Keyspaces (for Apache Cassandra) support for Cassandra v3.11 end of life schedule
Amazon Keyspaces (for Apache Cassandra) support for Cassandra v3.11 end of life schedule

Amazon Keyspaces (for Apache Cassandra) is a scalable, highly available, and managed Apache Cassandra-compatible database service. With Amazon Keyspaces, you can run your Cassandra workloads on AWS using the same Cassandra application code and developer tools that you use today. You don’t have to provision, patch, or manage servers, and you don’t have to install, […]

5 days, 1 hour ago @ aws.amazon.com
Alternatives to the Oracle flashback database feature in Amazon RDS for Oracle
Alternatives to the Oracle flashback database feature in Amazon RDS for Oracle

Customers may prefer to host their Oracle database workloads in a managed service such as Amazon Relational Database Service (Amazon RDS) for Oracle because of the benefits offered by managed services. However, there could be workloads that have dependencies on Oracle features that aren’t supported by Amazon RDS for Oracle. For example, the flashback database […]

5 days, 1 hour ago @ aws.amazon.com
Translate documents in real time with Amazon Translate
Translate documents in real time with Amazon Translate

A critical component of business success is the ability to connect with customers. Businesses today want to connect with their customers by offering their content across multiple languages in real time. For most customers, the content creation process is disconnected from the localization effort of translating content into multiple target languages. These disconnected processes delay […]

5 days, 2 hours ago @ aws.amazon.com
Scale your machine learning workloads on Amazon ECS powered by AWS Trainium instances
Scale your machine learning workloads on Amazon ECS powered by AWS Trainium instances

Running machine learning (ML) workloads with containers is becoming a common practice. Containers can fully encapsulate not just your training code, but the entire dependency stack down to the hardware libraries and drivers. What you get is an ML development environment that is consistent and portable. With containers, scaling on a cluster becomes much easier. […]

5 days, 2 hours ago @ aws.amazon.com
Host ML models on Amazon SageMaker using Triton: CV model with PyTorch backend
Host ML models on Amazon SageMaker using Triton: CV model with PyTorch backend

PyTorch is a machine learning (ML) framework based on the Torch library, used for applications such as computer vision and natural language processing. One of the primary reasons that customers are choosing a PyTorch framework is its simplicity and the fact that it’s designed and assembled to work with Python. PyTorch supports dynamic computational graphs, […]

5 days, 3 hours ago @ aws.amazon.com
Configure and use defaults for Amazon SageMaker resources with the SageMaker Python SDK
Configure and use defaults for Amazon SageMaker resources with the SageMaker Python SDK

The Amazon SageMaker Python SDK is an open-source library for training and deploying machine learning (ML) models on Amazon SageMaker. Enterprise customers in tightly controlled industries such as healthcare and finance set up security guardrails to ensure their data is encrypted and traffic doesn’t traverse the internet. To ensure the SageMaker training and deployment of […]

5 days, 3 hours ago @ aws.amazon.com
Configure Amazon OpenSearch Service for high availability
Configure Amazon OpenSearch Service for high availability

Amazon OpenSearch Service is a fully open-source search and analytics engine that securely unlocks real-time search, monitoring, and analysis of business and operational data for use cases like recommendation engines, ecommerce sites, and catalog search. To be successful in your business, you need your systems to be highly available and performant, minimizing downtime and avoiding […]

5 days, 5 hours ago @ aws.amazon.com
Trakstar unlocks new analytical opportunities for its HR customers with Amazon QuickSight
Trakstar unlocks new analytical opportunities for its HR customers with Amazon QuickSight

This is a guest post by Brian Kasen and Rebecca McAlpine from Trakstar, now a part of Mitratech. Trakstar, now a part of Mitratech, is a human resources (HR) software company that serves customers from small businesses and educational institutions to large enterprises, globally. Trakstar supercharges employee performance around pivotal moments in talent development. Our […]

5 days, 5 hours ago @ aws.amazon.com
Accelerate your learning towards AWS Certification exams with automated quiz generation using Amazon SageMaker foundations models
Accelerate your learning towards AWS Certification exams with automated quiz generation using Amazon SageMaker foundations models

Getting AWS Certified can help you propel your career, whether you’re looking to find a new role, showcase your skills to take on a new project, or become your team’s go-to expert. And because AWS Certification exams are created by experts in the relevant role or technical area, preparing for one of these exams helps […]

5 days, 5 hours ago @ aws.amazon.com
Optimize software development with Amazon CodeWhisperer
Optimize software development with Amazon CodeWhisperer

Businesses differentiate themselves by delivering new capabilities to their customers faster. They must leverage automation to accelerate their software development by optimizing code quality, improving performance, and ensuring their software meets security/compliance requirements. Trained on billions of lines of Amazon and open-source code, Amazon CodeWhisperer is an AI coding companion that helps developers write code […]

5 days, 22 hours ago @ aws.amazon.com
Azure
Last posted 1 week, 6 days ago
Introducing Microsoft Fabric: Data analytics for the era of AI
Introducing Microsoft Fabric: Data analytics for the era of AI

Today we are unveiling Microsoft Fabric—an end-to-end, unified analytics platform that brings together all the data and analytics tools that organizations need.

The post Introducing Microsoft Fabric: Data analytics for the era of AI appeared first on Azure Blog.

1 week, 6 days ago @ azure.microsoft.com
Build next-generation, AI-powered applications on Microsoft Azure
Build next-generation, AI-powered applications on Microsoft Azure

Welcome to Microsoft Build 2023—the event where we celebrate the developer community. This year, we’ll dive deep into the latest technologies across application development and AI that are enabling the next wave of innovation.

The post Build next-generation, AI-powered applications on Microsoft Azure appeared first on Azure Blog.

1 week, 6 days ago @ azure.microsoft.com
The synergy of market leaders: Exploring Microsoft and SAP’s game-changing collaboration
The synergy of market leaders: Exploring Microsoft and SAP’s game-changing collaboration

The partnership between Microsoft and SAP has enabled customers to accelerate their business transformation, improve resilience, minimize downtime, and prepare for more rapid growth.

The post The synergy of market leaders: Exploring Microsoft and SAP’s game-changing collaboration appeared first on Azure Blog.

2 weeks, 6 days ago @ azure.microsoft.com
Transforming containerized applications with Azure Container Storage—now in preview
Transforming containerized applications with Azure Container Storage—now in preview

Today, we are excited to announce the preview of Azure Container Storage, the industry’s first platform-managed container native storage service in the public cloud, providing end to end storage management and orchestration for stateful applications to run efficiently at scale on Azure.

The post Transforming containerized applications with Azure Container Storage—now in preview appeared first on Azure Blog.

3 weeks ago @ azure.microsoft.com
Optimize performance and strengthen security with Policy Analytics for Azure Firewall
Optimize performance and strengthen security with Policy Analytics for Azure Firewall

Network security policies are a critical component of modern IT environments, especially with the increasing adoption of cloud workloads. As workloads move to the cloud, network security policies like Azure Firewall policies evolve and adapt to the changing demands of the infrastructure. These policies can be updated multiple times a week, making it challenging for IT…

The post Optimize performance and strengthen security with Policy Analytics for Azure Firewall appeared first on Azure Blog.

3 weeks ago @ azure.microsoft.com
Microsoft Build 2023: Innovation through Microsoft commercial marketplace
Microsoft Build 2023: Innovation through Microsoft commercial marketplace

With Azure’s enterprise-grade security and built-in responsible AI, the rate of innovation is growing exponentially. As we look forward to Microsoft Build 2023, I am inspired by the innovation coming from our ISV partners and software as a service (SaaS) providers building on the Microsoft Cloud.

The post Microsoft Build 2023: Innovation through Microsoft commercial marketplace appeared first on Azure Blog.

3 weeks, 4 days ago @ azure.microsoft.com
Microsoft Azure security evolution: Embrace secure multitenancy, Confidential Compute, and Rust
Microsoft Azure security evolution: Embrace secure multitenancy, Confidential Compute, and Rust

In this installment, we will introduce our game-changing bets that will enable us to deliver industry-leading security architectures with built-in security for years to come, ensuring a secure cloud experience for our customers. The post Microsoft Azure security evolution: Embrace secure multitenancy, Confidential Compute, and Rust appeared first on Azure Blog.

3 weeks, 4 days ago @ azure.microsoft.com
Navigating the SPACE between productivity and developer happiness
Navigating the SPACE between productivity and developer happiness

To reflect our renewed focus on the overall developer experience, we are excited to share that we are rebranding Developer Velocity Lab to Developer Experience Lab. The post Navigating the SPACE between productivity and developer happiness appeared first on Azure Blog.

3 weeks, 4 days ago @ azure.microsoft.com
New and upcoming capabilities with Elastic Cloud (Elasticsearch)—An Azure Native ISV Service
New and upcoming capabilities with Elastic Cloud (Elasticsearch)—An Azure Native ISV Service

Microsoft and Elastic partnered together in 2020 to build an Elastic Cloud (Elasticsearch)—An Azure Native ISV Service to create cloud native deeply integrated experiences for all Azure and Elastic customers to power their digital transformation. The post New and upcoming capabilities with Elastic Cloud (Elasticsearch)—An Azure Native ISV Service appeared first on Azure Blog.

3 weeks, 5 days ago @ azure.microsoft.com
What’s new with Azure Files
What’s new with Azure Files

In this blog we will talk about how SAP ECS uses Azure Files NFS shares as part of their default deployment and we’ll also announce some of the big NFS feature we’ve been working over the past year including: new 99.99 percent SLA, snapshot support, and nconnect.

The post What’s new with Azure Files appeared first on Azure Blog.

3 weeks, 5 days ago @ azure.microsoft.com
Insights from the 2023 Open Confidential Computing Conference
Insights from the 2023 Open Confidential Computing Conference

I had the opportunity to participate in this year's Open Confidential Computing Conference (OC3), hosted by our software partner, Edgeless Systems. The panel featured some of the industry's most respected technology leaders.

The post Insights from the 2023 Open Confidential Computing Conference appeared first on Azure Blog and Updates.

4 weeks ago @ azure.microsoft.com
Insights from the 2023 Open Confidential Computing Conference
Insights from the 2023 Open Confidential Computing Conference

I had the opportunity to participate in this year's Open Confidential Computing Conference (OC3), hosted by our software partner, Edgeless Systems. This year's event was particularly noteworthy due to a panel discussion on the impact and future of confidential computing. The panel featured some of the industry's most respected technology leaders including Greg Lavender, Chief Technical Officer at Intel, Ian Buck, Vice President of Hyperscale and HPC at NVIDIA, and Mark Papermaster, Chief Technical Officer at AMD.

4 weeks ago @ azure.microsoft.com
Microsoft Cost Management updates—April 2023
Microsoft Cost Management updates—April 2023

April introduces scheduled cost alerts for Azure Government and a wealth of tips and new ways to save with Microsoft 365 and Azure, plus 8 new and updated docs. Enjoy!

The post Microsoft Cost Management updates—April 2023 appeared first on Azure Blog and Updates.

1 month ago @ azure.microsoft.com
Preparing for future health emergencies with Azure HPC
Preparing for future health emergencies with Azure HPC

COVID-19 has forever changed how nations function in the globally interconnected economy. To this day, it continues to affect and shape how countries respond to health emergencies.

The post Preparing for future health emergencies with Azure HPC appeared first on Azure Blog and Updates.

1 month ago @ azure.microsoft.com
Microsoft Cost Management updates—April 2023
Microsoft Cost Management updates—April 2023

April introduces scheduled cost alerts for Azure Government and a wealth of tips and new ways to save with Microsoft 365 and Azure, plus 8 new and updated docs. Enjoy!

1 month ago @ azure.microsoft.com