Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
115 commits
Select commit Hold shift + click to select a range
1eec8f9
Merge pull request #23 from Netflix/master
v1r3n Oct 6, 2017
e787435
Merge pull request #24 from Netflix/master
v1r3n Oct 6, 2017
0e98adf
Merge pull request #25 from Netflix/master
v1r3n Oct 6, 2017
f3dd104
ignoring IDEA files
robzienert Dec 4, 2017
b9caecc
ignoring .gradle dir
robzienert Dec 4, 2017
a022a1e
refactoring queue implementations to accept a Clock
robzienert Dec 4, 2017
29b9efc
Merge pull request #26 from robzienert/clock-refactor
v1r3n Dec 18, 2017
267567b
refactored for the new pipeline based queues
v1r3n Feb 11, 2018
6335430
fixed https://github.com/Netflix/dyno-queues/issues/19
v1r3n Feb 11, 2018
647f812
v2 recipe that uses redis pipes
Mar 16, 2018
6d9ce47
update dyno version
Mar 16, 2018
e74db61
Apply license header to the files.
Mar 17, 2018
309ef57
more tests and implement long polling for the v2 recipe
Mar 18, 2018
a1a8e39
refactored tests so the queue names are distinct across tests
Mar 18, 2018
3ac3dbe
javadocs
Mar 18, 2018
efb295a
use hashtags to separate messages per shard
Mar 20, 2018
3329399
Fixed typo
michel-zededa Apr 20, 2018
9232b8d
Merge pull request #28 from michel-zededa/dev
v1r3n May 2, 2018
e92d2aa
update dyno version to the latest stable
ipapapa Sep 20, 2018
4f008f7
update dyno version to the latest stable
ipapapa Sep 20, 2018
d1185bd
Gradle, nebula and REdis pipeline
ipapapa Oct 1, 2018
c3324d7
Merge pull request #33 from Netflix/cde-updates
ipapapa Oct 2, 2018
154bb2d
Custom sharding strategy
KowalczykBartek Oct 8, 2018
82e5ae6
fix bundle.gradle to pick up RCs and upgrade dyno-core and dyno-jedis…
Oct 9, 2018
ec33a3a
Review fix - test and improvments
KowalczykBartek Oct 10, 2018
b8e5168
change custom shard strategy logic
KowalczykBartek Oct 12, 2018
8af9b44
Remove some unused lines (solves #29)
Ricool06 Oct 13, 2018
c8aa2cf
Merge pull request #35 from Ricool06/dev
ipapapa Oct 15, 2018
0eaebf6
Make v2 queue more safe
KowalczykBartek Oct 19, 2018
adebe95
Merge recent changes from develop
KowalczykBartek Oct 19, 2018
cb6ca82
Merge pull request #34 from KowalczykBartek/dev
smukil Nov 7, 2018
bc8569e
Add information about operation and key in exception strings
smukil Nov 7, 2018
46d5cb2
Merge pull request #36 from smukil/log_key_on_failure
smukil Nov 15, 2018
7f93610
Updates dyno to latest stable release
davidwadden Dec 13, 2018
f781c48
Merge pull request #37 from davidwadden/dyno-upgrade
kishorekasi Dec 13, 2018
f8b68a8
returning shard directly without assigning to variable
saurabhdev7933 Mar 1, 2019
cf18218
Adding simple demo for queues
rsrinivasanNetflix Apr 30, 2019
2a2a3e4
Create ensure() API
smukil Apr 30, 2019
d090cb2
Trying to fix flaky test
rsrinivasanNetflix Apr 30, 2019
a69810c
Fixes from comments
rsrinivasanNetflix Apr 30, 2019
095f394
Added doc comment
rsrinivasanNetflix May 2, 2019
3f669eb
Merge pull request #39 from smukil/ensure_api
smukil May 2, 2019
8bd1669
Merge pull request #40 from Netflix/feature/addSimpleDemo
rsrinivasanNetflix May 3, 2019
e080716
Create containsPredicate() API
smukil May 3, 2019
d3c5506
Cast from 'JedisCommands' to 'DynoJedisClient' instead of 'Jedis' in
smukil May 7, 2019
6506c4d
DynoShardSupplier should return consistent results through its APIs
smukil May 7, 2019
3aa2d06
Merge pull request #42 from smukil/containsPredicate_api_with_fix
smukil May 7, 2019
016be83
Add a demo for the V1 queue
smukil May 7, 2019
2fd3a65
Merge pull request #43 from smukil/fix_dynoShardSupplier
smukil May 7, 2019
16b89be
Merge pull request #38 from saurabhdev7933/return-shard-no-variable
smukil May 8, 2019
c432aef
Use logger instead of System.out in demo
smukil May 8, 2019
4f4ca2b
Merge pull request #44 from smukil/v1_queue_demo
smukil May 8, 2019
cbafb5c
Fix issue with javadoc not building
rsrinivasanNetflix May 15, 2019
d37130c
Merge pull request #46 from Netflix/bugfix/buildJavadoc
rsrinivasanNetflix May 15, 2019
ce4b87e
Auto formatting and fixing host to shard map
rsrinivasanNetflix May 28, 2019
1cde55b
Merge pull request #47 from Netflix/feature/autoFormatFixHostToShard
rsrinivasanNetflix May 31, 2019
270ead1
Bumping up the dyno dependency
rsrinivasanNetflix Jun 4, 2019
b5443e3
Fix for jedis version change
rsrinivasanNetflix Jun 5, 2019
45c5b13
Fixing issues with moving to the new jedis
rsrinivasanNetflix Jun 5, 2019
2fb9f8b
Merge pull request #48 from Netflix/feature/bumpDynoVersion
rsrinivasanNetflix Jun 5, 2019
39d99db
Changing api for queuebuilder to be simpler
rsrinivasanNetflix Jun 12, 2019
02b16e6
Fix from comments
rsrinivasanNetflix Jun 21, 2019
0133961
Pin to dyno 1.6.7
rsrinivasanNetflix Jun 21, 2019
64adfda
Switch Travis CI from using Oracle JDK 8 to Open JDK 8
smukil Aug 16, 2019
dc3710a
Merge pull request #53 from smukil/switch_travisci_to_openjdk
smukil Aug 16, 2019
71a529b
Update dyno dependency to 1.6.9
smukil Aug 16, 2019
be985f3
Create getMsgWithPredicate() API
smukil Aug 29, 2019
f481335
getMsgWithPredicate() should use nonQuorumConn
smukil Aug 30, 2019
6bf4bc2
Pin to dyno 1.7.0
smukil Sep 5, 2019
79f246a
Update APIs to compile with dyno-1.7.1
smukil Aug 26, 2019
251f94e
Add a popWithMsgId() API
smukil Sep 17, 2019
761e231
Allow Lua pattern matching in containsPredicate() and getMsgWithPredi…
smukil Sep 17, 2019
52f79ee
remove() may return 'false' even though it removed the element
smukil Sep 17, 2019
bb124c9
Add logging for processUnacks()
smukil Sep 18, 2019
7ccd0b9
Make peekIds() use nonQuorumConn.
smukil Sep 18, 2019
34fea81
processUnacks() should use nonQuorumConn to look over a range in the
smukil Sep 18, 2019
964f57a
Add unsafePeekAllShards() and unsafePopAllShards() functions
smukil May 9, 2019
7b1b4ea
unsafePopAllShards() should use the prefetched IDs from the unsafe Map
smukil Sep 19, 2019
41376a4
Clarify ordering guarantees with unsafePeekAllShards()
smukil Sep 19, 2019
9f08682
Add a new ShardSupplier to address issues with DynoShardSupplier
smukil Sep 20, 2019
75f9c4b
Allow containsPredicate()/getMsgWithPredicate() to filter on local shard
smukil Sep 23, 2019
6975b46
Add unsafePopWithMsgIdAllShards() API
smukil Sep 23, 2019
c3fc231
Make debug message clearer in popWithMsgId()
smukil Oct 2, 2019
d9cfc62
Use object form of Long for zrank() return type
smukil Oct 2, 2019
d84fd5c
Move debug messages from DEBUG to WARN
smukil Oct 8, 2019
2218bb3
popWithMsgId() and unsafePopWithMsgId() should report metrics
smukil Oct 11, 2019
9ccba78
Pin to dyno-1.7.2
smukil Oct 21, 2019
8824bfd
Pin to dyno-1.7.2-rc2
smukil Oct 22, 2019
a6f5592
unsafePopWithMsgId() shouldn't spam the logs
smukil Oct 23, 2019
749596c
Add popMsgWithPredicate() API
smukil Oct 29, 2019
d4305d4
Add a atomic bulkPop() API
smukil Nov 11, 2019
5089a7c
Add unsafeBulkPop() and localGet()
smukil Nov 14, 2019
6d8a91a
atomicBulkPopHelper() should cast to Message before returning
smukil Nov 15, 2019
84325c0
Make popWithMsgPredicate() obey queue priority
smukil Nov 15, 2019
ad6c57f
Fix null return from popMsgWithPredicateObeyPriority
smukil Nov 15, 2019
12d242a
Add atomicProcessUnacks() and getAllMessages()
smukil Jan 17, 2020
ee40066
JedisMock cannot be cast to DynoJedisClient
smukil Jan 17, 2020
3da0ec1
Make popMsgWithPredicateObeyPriority() check if hash exists
smukil Jan 21, 2020
de3787f
Add an atomicRemove() API
smukil Jan 22, 2020
5b25b69
Add findStaleMessages() API
smukil Jan 23, 2020
03af6f5
Make findStaleMessages() return stale messages from all shards
smukil Jan 25, 2020
7f8e681
Update README.md
smukil Aug 25, 2020
6722cb2
Comment out Global properties from demo.properties
smukil Nov 17, 2020
1089056
Update build status location
smukil Nov 17, 2020
826888f
Upgrade nebula.netflixoss to replace bintray publication and update T…
rpalcolea Mar 4, 2021
8a36294
Merge pull request #101 from Netflix/replace-bintray
rpalcolea Mar 9, 2021
e9a3eea
Replace JCenter with Maven Central
rpalcolea Mar 18, 2021
88805bf
Merge pull request #102 from Netflix/replace-jcenter
rpalcolea Mar 18, 2021
e20195a
Rotate publishing credentials
rpalcolea Apr 21, 2021
2b193c1
rotate TravisCI secrets
rpalcolea Aug 17, 2021
07bcd27
Remove TravisCI and use Github Actions
rpalcolea Aug 17, 2021
a920a8e
Merge pull request #104 from Netflix/use-github-actions
rpalcolea Aug 17, 2021
d4a5dfa
refactor: Change GitHub Action version to address deprecation
rpalcolea Dec 10, 2024
6104363
Upgrade Gradle
rpalcolea Dec 10, 2024
3ba0d2a
Fix lingering messages in the hash after remove() is invoked on the d…
ashah4 Jun 9, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 58 additions & 0 deletions .github/workflows/nebula-ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
name: "CI"
on:
push:
branches:
- '*'
tags-ignore:
- '*'
pull_request:

jobs:
build:
runs-on: ubuntu-latest
services:
redis:
image: redis
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 6379:6379
strategy:
matrix:
# test against JDK 8
java: [ 8 ]
name: CI with Java ${{ matrix.java }}
steps:
- uses: actions/checkout@v4
- run: |
git config --global user.name "Netflix OSS Maintainers"
git config --global user.email "netflixoss@netflix.com"
- name: Setup jdk
uses: actions/setup-java@v1
with:
java-version: ${{ matrix.java }}
- uses: actions/cache@v4
id: gradle-cache
with:
path: ~/.gradle/caches
key: ${{ runner.os }}-gradle-${{ hashFiles('**/gradle/dependency-locks/*.lockfile') }}
restore-keys: |
- ${{ runner.os }}-gradle-
- uses: actions/cache@v4
id: gradle-wrapper-cache
with:
path: ~/.gradle/wrapper
key: ${{ runner.os }}-gradlewrapper-${{ hashFiles('gradle/wrapper/*') }}
restore-keys: |
- ${{ runner.os }}-gradlewrapper-
- name: Build with Gradle
run: ./gradlew --info --stacktrace build
env:
CI_NAME: github_actions
CI_BUILD_NUMBER: ${{ github.sha }}
CI_BUILD_URL: 'https://github.com/${{ github.repository }}'
CI_BRANCH: ${{ github.ref }}
COVERALLS_REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
64 changes: 64 additions & 0 deletions .github/workflows/nebula-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
name: "Publish candidate/release to NetflixOSS and Maven Central"
on:
push:
tags:
- v*.*.*
- v*.*.*-rc.*
release:
types:
- published

jobs:
build:
runs-on: ubuntu-latest
services:
redis:
image: redis
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 6379:6379
steps:
- uses: actions/checkout@v4
- run: |
git config --global user.name "Netflix OSS Maintainers"
git config --global user.email "netflixoss@netflix.com"
- name: Setup jdk 8
uses: actions/setup-java@v1
with:
java-version: 1.8
- uses: actions/cache@v4
id: gradle-cache
with:
path: ~/.gradle/caches
key: ${{ runner.os }}-gradle-${{ hashFiles('**/gradle/dependency-locks/*.lockfile') }}
restore-keys: |
- ${{ runner.os }}-gradle-
- uses: actions/cache@v4
id: gradle-wrapper-cache
with:
path: ~/.gradle/wrapper
key: ${{ runner.os }}-gradlewrapper-${{ hashFiles('gradle/wrapper/*') }}
restore-keys: |
- ${{ runner.os }}-gradlewrapper-
- name: Publish candidate
if: contains(github.ref, '-rc.')
run: ./gradlew --info --stacktrace -Prelease.useLastTag=true candidate
env:
NETFLIX_OSS_SIGNING_KEY: ${{ secrets.ORG_SIGNING_KEY }}
NETFLIX_OSS_SIGNING_PASSWORD: ${{ secrets.ORG_SIGNING_PASSWORD }}
NETFLIX_OSS_REPO_USERNAME: ${{ secrets.ORG_NETFLIXOSS_USERNAME }}
NETFLIX_OSS_REPO_PASSWORD: ${{ secrets.ORG_NETFLIXOSS_PASSWORD }}
- name: Publish release
if: (!contains(github.ref, '-rc.'))
run: ./gradlew --info -Prelease.useLastTag=true final
env:
NETFLIX_OSS_SONATYPE_USERNAME: ${{ secrets.ORG_SONATYPE_USERNAME }}
NETFLIX_OSS_SONATYPE_PASSWORD: ${{ secrets.ORG_SONATYPE_PASSWORD }}
NETFLIX_OSS_SIGNING_KEY: ${{ secrets.ORG_SIGNING_KEY }}
NETFLIX_OSS_SIGNING_PASSWORD: ${{ secrets.ORG_SIGNING_PASSWORD }}
NETFLIX_OSS_REPO_USERNAME: ${{ secrets.ORG_NETFLIXOSS_USERNAME }}
NETFLIX_OSS_REPO_PASSWORD: ${{ secrets.ORG_NETFLIXOSS_PASSWORD }}
50 changes: 50 additions & 0 deletions .github/workflows/nebula-snapshot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name: "Publish snapshot to NetflixOSS and Maven Central"

on:
push:
branches:
- master

jobs:
build:
runs-on: ubuntu-latest
services:
redis:
image: redis
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 6379:6379
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- run: |
git config --global user.name "Netflix OSS Maintainers"
git config --global user.email "netflixoss@netflix.com"
- name: Set up JDK
uses: actions/setup-java@v1
with:
java-version: 8
- uses: actions/cache@v4
id: gradle-cache
with:
path: |
~/.gradle/caches
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle') }}
- uses: actions/cache@v4
id: gradle-wrapper-cache
with:
path: |
~/.gradle/wrapper
key: ${{ runner.os }}-gradlewrapper-${{ hashFiles('gradle/wrapper/*') }}
- name: Build
run: ./gradlew build snapshot
env:
NETFLIX_OSS_SIGNING_KEY: ${{ secrets.ORG_SIGNING_KEY }}
NETFLIX_OSS_SIGNING_PASSWORD: ${{ secrets.ORG_SIGNING_PASSWORD }}
NETFLIX_OSS_REPO_USERNAME: ${{ secrets.ORG_NETFLIXOSS_USERNAME }}
NETFLIX_OSS_REPO_PASSWORD: ${{ secrets.ORG_NETFLIXOSS_PASSWORD }}
13 changes: 13 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,16 @@ dyno-queues-core/build
build
redis-3.0.7/
redis-3.0.7.tar.gz

*.iml
.idea
.gradle
.classpath
.project


dyno-queues-core/out/
dyno-queues-redis/out/

# publishing secrets
secrets/signing-key
34 changes: 0 additions & 34 deletions .travis.yml

This file was deleted.

6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
## DISCLAIMER: THIS PROJECT IS NO LONGER ACTIVELY MAINTAINED



# Dyno Queues
[![Build Status](https://travis-ci.org/Netflix/dyno-queues.svg)](https://travis-ci.org/Netflix/dyno-queues)
[![Build Status](https://travis-ci.com/Netflix/dyno-queues.svg)](https://travis-ci.com/Netflix/dyno-queues)
[![Dev chat at https://gitter.im/Netflix/dynomite](https://badges.gitter.im/Netflix/dynomite.svg)](https://gitter.im/Netflix/dynomite?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)

Dyno Queues is a recipe that provides task queues utilizing [Dynomite](https://github.com/Netflix/dynomite).
Expand Down
29 changes: 17 additions & 12 deletions build.gradle
Original file line number Diff line number Diff line change
@@ -1,16 +1,19 @@
buildscript {

repositories {
jcenter()
repositories {
mavenCentral()
maven {
url = 'https://plugins.gradle.org/m2'
}
}

dependencies {
classpath 'com.netflix.nebula:gradle-aggregate-javadocs-plugin:2.2.+'
classpath 'com.netflix.nebula:gradle-extra-configurations-plugin:3.0.3'
classpath 'com.netflix.nebula:gradle-aggregate-javadocs-plugin:3.0.1'
classpath 'com.netflix.nebula:gradle-extra-configurations-plugin:4.0.1'
}
}

plugins {
id 'nebula.netflixoss' version '3.6.0'
id 'com.netflix.nebula.netflixoss' version '11.5.0'
}

// Establish version and status
Expand All @@ -19,18 +22,20 @@ ext.githubProjectName = rootProject.name // Change if github project name is not
apply plugin: 'project-report'

subprojects {

apply plugin: 'nebula.netflixoss'
apply plugin: 'java'
apply plugin: 'idea'
apply plugin: 'eclipse'
apply plugin: 'java-library'
apply plugin: 'project-report'

sourceCompatibility = 1.8
targetCompatibility = 1.8

repositories {
jcenter()
repositories {
mavenCentral()

// oss-candidate for -rc.* verions:
maven {
url "https://dl.bintray.com/netflixoss/oss-candidate"
}
}

group = "com.netflix.${githubProjectName}"
Expand Down
22 changes: 0 additions & 22 deletions buildViaTravis.sh

This file was deleted.

5 changes: 3 additions & 2 deletions dyno-queues-core/build.gradle
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@

dependencies {
testCompile "junit:junit:4.11"
}
api 'com.netflix.dyno:dyno-core:1.7.2-rc2'
testImplementation "junit:junit:4.11"
}
Loading