Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generic Key generators #214

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Generic Key generators #214

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions ndbench-api/src/main/java/com/netflix/ndbench/api/plugin/NdBenchAbstractClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
*
* @author vchella, pencal
*/
public interface NdBenchAbstractClient<W> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

while we are in here, lets discuss the naming of parameter W. W stands for write because it is the type returned by writeSingle but it is not descriptive of what the type is. Can we rename W to PostImageType or am I thinking about it too much?

public interface NdBenchAbstractClient<K, W> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please document the addition of parameter K. Also, is it OK not to narrow down the expected superclass/interfaces for K?


/**
* Initialize the client
Expand All @@ -47,31 +47,31 @@ public interface NdBenchAbstractClient<W> {
* @return
* @throws Exception
*/
String readSingle(final String key) throws Exception;
String readSingle(final K key) throws Exception;

/**
* Perform a bulk read operation given the list of keys
*
* @return
* @throws Exception
*/
List<String> readBulk(final List<String> keys) throws Exception;
List<String> readBulk(final List<K> keys) throws Exception;

/**
* Perform a single write operation
*
* @return
* @throws Exception
*/
W writeSingle(final String key) throws Exception;
W writeSingle(final K key) throws Exception;

/**
* Perform bulk write operation given the list of keys
*
* @param keys
* @return
*/
List<W> writeBulk(final List<String> keys) throws Exception;
List<W> writeBulk(final List<K> keys) throws Exception;

/**
* shutdown the client
Expand Down
10 changes: 5 additions & 5 deletions ndbench-api/src/main/java/com/netflix/ndbench/api/plugin/NdBenchBaseClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,26 +23,26 @@
/**
* @author vchella, pencal
*/
public abstract class NdBenchBaseClient implements NdBenchClient {
public abstract class NdBenchBaseClient<K,W> implements NdBenchClient<K,W> {

@Override
public String readSingle(final String key) throws Exception {
public String readSingle(final K key) throws Exception {
return null;
}


@Override
public String writeSingle(final String key) throws Exception {
public W writeSingle(final K key) throws Exception {
return null;
}

@Override
public List<String> readBulk(final List<String> keys) throws Exception {
public List<String> readBulk(final List<K> keys) throws Exception {
return null;
}

@Override
public List<String> writeBulk(final List<String> keys) throws Exception {
public List<W> writeBulk(final List<K> keys) throws Exception {
return null;
}

Expand Down
10 changes: 5 additions & 5 deletions ndbench-api/src/main/java/com/netflix/ndbench/api/plugin/NdBenchClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
/**
* @author vchella, pencal
*/
public interface NdBenchClient extends NdBenchAbstractClient<String> {
public interface NdBenchClient<K,W> extends NdBenchAbstractClient<K, W> {

/**
* Initialize the client
Expand All @@ -35,21 +35,21 @@ public interface NdBenchClient extends NdBenchAbstractClient<String> {
* @return
* @throws Exception
*/
String readSingle(final String key) throws Exception;
String readSingle(final K key) throws Exception;

/**
* Perform a single write operation
* @return
* @throws Exception
*/
String writeSingle(final String key) throws Exception;
W writeSingle(final K key) throws Exception;

/**
* Perform a bulk read operation
* @return
* @throws Exception
*/
default List<String> readBulk(final List<String> keys) throws Exception {
default List<String> readBulk(final List<K> keys) throws Exception {
throw new UnsupportedOperationException("bulk operation is not supported");
}

Expand All @@ -58,7 +58,7 @@ default List<String> readBulk(final List<String> keys) throws Exception {
* @return
* @throws Exception
*/
default List<String> writeBulk(final List<String> keys) throws Exception {
default List<W> writeBulk(final List<K> keys) throws Exception {
throw new UnsupportedOperationException("bulk operation is not supported");
}

Expand Down
2 changes: 1 addition & 1 deletion ...nch-cass-plugins/src/main/java/com/netflix/ndbench/plugin/cass/CJavaDriverBasePlugin.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
* @author vchella
* @author Alexander Patrikalakis
*/
public abstract class CJavaDriverBasePlugin<C extends CassandraConfigurationBase> implements NdBenchClient {
public abstract class CJavaDriverBasePlugin<C extends CassandraConfigurationBase> implements NdBenchClient<String,String> {

private static final Logger logger = LoggerFactory.getLogger(CJavaDriverBasePlugin.class);
protected static final String ResultOK = "Ok";
Expand Down
2 changes: 1 addition & 1 deletion ndbench-cass-plugins/src/main/java/com/netflix/ndbench/plugin/cass/CassAstyanaxPlugin.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
*/
@Singleton
@NdBenchClientPlugin("CassAstyanaxPlugin")
public class CassAstyanaxPlugin implements NdBenchClient {
public class CassAstyanaxPlugin implements NdBenchClient<String,String> {
private static final Logger logger = LoggerFactory.getLogger(CassAstyanaxPlugin.class);
private static final String ResultOK = "Ok";
private static final String CacheMiss = null;
Expand Down
2 changes: 1 addition & 1 deletion ...s/src/main/java/com/netflix/ndbench/plugin/elassandra/ElassandraCassJavaDriverPlugin.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@
*/
@Singleton
@NdBenchClientPlugin("ElassandraCassJavaDriverPlugin")
public class ElassandraCassJavaDriverPlugin implements NdBenchClient{
public class ElassandraCassJavaDriverPlugin implements NdBenchClient<String,String>{
private static final Logger logger = LoggerFactory.getLogger(ElassandraCassJavaDriverPlugin.class);
private static final String ResultOK = "Ok";
private static final String CacheMiss = null;
Expand Down
28 changes: 14 additions & 14 deletions ndbench-core/src/main/java/com/netflix/ndbench/core/DataBackfill.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,19 +60,19 @@ public DataBackfill(IConfiguration config) {
this.config = config;
}

public void backfill(final NdBenchAbstractClient<?> client) throws Exception {
public void backfill(final NdBenchAbstractClient<?,?>client) throws Exception {
backfill(client, new NormalBackfill());
}

public void conditionalBackfill(final NdBenchAbstractClient<?> client) throws Exception {
public void conditionalBackfill(final NdBenchAbstractClient<?,?>client) throws Exception {
backfill(client, new ConditionalBackfill());
}

public void verifyBackfill(final NdBenchAbstractClient<?> client) throws Exception {
public void verifyBackfill(final NdBenchAbstractClient<?,?>client) throws Exception {
backfill(client, new VerifyBackfill());
}

private void backfill(final NdBenchAbstractClient<?> client, final BackfillOperation backfillOperation) throws Exception {
private void backfill(final NdBenchAbstractClient<?,?>client, final BackfillOperation backfillOperation) throws Exception {

long start = System.currentTimeMillis();

Expand All @@ -83,11 +83,11 @@ private void backfill(final NdBenchAbstractClient<?> client, final BackfillOpera
logger.info("Backfiller latch done! in " + (System.currentTimeMillis() - start) + " ms");
}

public void backfillAsync(final NdBenchAbstractClient<?> client) {
public void backfillAsync(final NdBenchAbstractClient<?,?>client) {
backfillAsync(client, new NormalBackfill());
}

private void backfillAsync(final NdBenchAbstractClient<?> client, final BackfillOperation backfillOperation) {
private void backfillAsync(final NdBenchAbstractClient<?,?>client, final BackfillOperation backfillOperation) {
stop.set(false);

//Default #Cores*4 so that we can keep the CPUs busy even while waiting on I/O
Expand Down Expand Up @@ -166,23 +166,23 @@ public boolean getIsBackfillRunning() {
return false; //Never started
}

private interface BackfillOperation {
String process(final NdBenchAbstractClient<?> client, final String key) throws Exception;
private interface BackfillOperation<K,W> {
String process(final NdBenchAbstractClient<K,W> client, final K key) throws Exception;
}

private class NormalBackfill implements BackfillOperation {
private class NormalBackfill<K,W> implements BackfillOperation<K,W> {

@Override
public String process(NdBenchAbstractClient<?> client, String key) throws Exception {
public String process(NdBenchAbstractClient<K,W> client, K key) throws Exception {
Object result = client.writeSingle(key);
return result == null ? "<null>" : result.toString();
}
}

private class ConditionalBackfill implements BackfillOperation {
private class ConditionalBackfill<K,W> implements BackfillOperation<K,W> {

@Override
public String process(NdBenchAbstractClient<?> client, String key) throws Exception {
public String process(NdBenchAbstractClient<K,W> client, K key) throws Exception {
String result = client.readSingle(key);
if (result == null) {
missCount.incrementAndGet();
Expand All @@ -193,10 +193,10 @@ public String process(NdBenchAbstractClient<?> client, String key) throws Except
}
}

private class VerifyBackfill implements BackfillOperation {
private class VerifyBackfill<K,W> implements BackfillOperation<K,W> {

@Override
public String process(NdBenchAbstractClient<?> client, String key) throws Exception {
public String process(NdBenchAbstractClient<K,W> client, K key) throws Exception {
Object result = client.writeSingle(key);
String value = client.readSingle(key);
if (value == null) {
Expand Down
8 changes: 4 additions & 4 deletions ndbench-core/src/main/java/com/netflix/ndbench/core/NdBenchClientFactory.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,15 +29,15 @@
@Singleton
public class NdBenchClientFactory {

private Map<String, NdBenchAbstractClient<?>> clientMap;
private Map<String, NdBenchAbstractClient<?,?>> clientMap;

@Inject
public NdBenchClientFactory(Map<String, NdBenchAbstractClient<?>> driverMap) {
public NdBenchClientFactory(Map<String, NdBenchAbstractClient<?,?>> driverMap) {
this.clientMap = driverMap;
}

public NdBenchAbstractClient<?> getClient(String clientName) {
NdBenchAbstractClient<?> client = clientMap.get(clientName);
public NdBenchAbstractClient<?,?> getClient(String clientName) {
NdBenchAbstractClient<?,?> client = clientMap.get(clientName);
if (client == null) {
throw new RuntimeException("Client not found: " + clientName);
}
Expand Down
50 changes: 19 additions & 31 deletions ndbench-core/src/main/java/com/netflix/ndbench/core/NdBenchDriver.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,8 +75,8 @@ public class NdBenchDriver {
private final AtomicReference<ExecutorService> timerRef = new AtomicReference<ExecutorService>(null);
private final RPSCount rpsCount;

private final AtomicReference<NdBenchAbstractClient<?>> clientRef =
new AtomicReference<NdBenchAbstractClient<?>>(null);
private final AtomicReference<NdBenchAbstractClient<?,?>> clientRef =
new AtomicReference<NdBenchAbstractClient<?,?>>(null);

private final AtomicReference<KeyGenerator> keyGeneratorWriteRef = new AtomicReference<>(null);
private final AtomicReference<KeyGenerator> keyGeneratorReadRef = new AtomicReference<>(null);
Expand Down Expand Up @@ -334,19 +334,7 @@ public void stopOperation(AtomicReference<ExecutorService> tpRef) {
logger.info("Threadpool has terminated!");
}

public interface NdBenchOperation {
boolean process(NdBenchDriver driver,
NdBenchMonitor monitor,
List<String> keys,
AtomicReference<RateLimiter> rateLimiter,
boolean isAutoTuneEnabled);

boolean isReadType();

boolean isWriteType();
}

public void init(NdBenchAbstractClient<?> client) throws Exception {
public void init(NdBenchAbstractClient<?,?> client) throws Exception {
if (!clientInited.get()) {
try {
if (clientInited.compareAndSet(false, true)) {
Expand Down Expand Up @@ -434,22 +422,22 @@ public void shutdownClient() throws Exception {
}
}

public String readSingle(String key) throws Exception {
try {
return clientRef.get().readSingle(key);
} catch (Exception e) {
logger.error("FAILED readSingle ", e);
throw e;
}
}


public String writeSingle(String key) throws Exception {
Object result = clientRef.get().writeSingle(key);
return result == null ? "<null>" : result.toString();
}

public NdBenchAbstractClient<?> getClient() {
// public String readSingle(String key) throws Exception {
// try {
// return clientRef.get().readSingle(key);
// } catch (Exception e) {
// logger.error("FAILED readSingle ", e);
// throw e;
// }
// }
//
//
// public String writeSingle(String key) throws Exception {
// Object result = clientRef.get().writeSingle(key);
// return result == null ? "<null>" : result.toString();
// }

public NdBenchAbstractClient<?,?> getClient() {
return clientRef.get();
}

Expand Down
20 changes: 20 additions & 0 deletions ndbench-core/src/main/java/com/netflix/ndbench/core/NdBenchOperation.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
package com.netflix.ndbench.core;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a license header


import java.util.List;
import java.util.concurrent.atomic.AtomicReference;

import com.google.common.util.concurrent.RateLimiter;

import com.netflix.ndbench.api.plugin.NdBenchMonitor;

public interface NdBenchOperation<K> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add a classdoc. also, is there some functional interface we could make this interface extend? like for example Function or BiFunction?

boolean process(NdBenchDriver driver,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add javadoc on the spec

NdBenchMonitor monitor,
List<K> keys,
AtomicReference<RateLimiter> rateLimiter,
boolean isAutoTuneEnabled);

boolean isReadType();

boolean isWriteType();
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a newline at the end of file

6 changes: 3 additions & 3 deletions ndbench-core/src/main/java/com/netflix/ndbench/core/defaultimpl/NdBenchClientModule.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@
public class NdBenchClientModule extends AbstractModule {
private static final Logger logger = LoggerFactory.getLogger(NdBenchClientModule.class);

private MapBinder<String, NdBenchAbstractClient<?>> maps;
private MapBinder<String, NdBenchAbstractClient<?,?>> maps;

private String getAnnotationValue(Class<?> ndBenchClientImpl) {
String name = ndBenchClientImpl.getName();
Expand All @@ -61,15 +61,15 @@ private <T> void installNdBenchClientPlugin(Class<?> ndBenchClientImpl) {
if (maps == null) {
TypeLiteral<String> stringTypeLiteral = new TypeLiteral<String>() {
};
TypeLiteral<NdBenchAbstractClient<?>> ndbClientTypeLiteral = (new TypeLiteral<NdBenchAbstractClient<?>>() {
TypeLiteral<NdBenchAbstractClient<?,?>> ndbClientTypeLiteral = (new TypeLiteral<NdBenchAbstractClient<?,?>>() {
});
maps = MapBinder.newMapBinder(binder(), stringTypeLiteral, ndbClientTypeLiteral);
}

String name = getAnnotationValue(ndBenchClientImpl);


maps.addBinding(name).to((Class<? extends NdBenchAbstractClient<?>>) ndBenchClientImpl);
maps.addBinding(name).to((Class<? extends NdBenchAbstractClient<?,?>>) ndBenchClientImpl);
}

@Override
Expand Down
Loading