12 October 2016

Using Java 8 Optional and reduce null checking


I am a big fan with the new streams API of Java 8, and I hope it would be used every where in the language.

For example, I hope that, Instead of CompletableFuture interface has  a method named thenApply I hope the name would be "map" instead. and for method like thenAccept, I hope the name would be "forEach" instead.

The point is, thenApply method in the CompleteableFuture interface is more or less a mapping operation, that takes the output of the step before it as input and generates a new output.

Also, theAccept is kind of consumer operation, like forEach, which will will take the output of the previous step as an input and consumes it without generating any output.

And if we talk about the Optional object in Java 8, although I don't have much more functional operations, but he have good enough to allow us to use it to reduce null checking.

The Optional Object has a map and filter (besides its regular methods such as orElseXXX methods).

Here's One example of its usage:

Suppose we have this Object structure that returned from some webservice:

                  static class Result {
Departement departement;
}


static class Departement{
List<Employee> employees;
}


static class Employee{
String name;
}

Then suppose our webservice return the result as one of these 4 functions:

       static Result simulateWsResult1() {
Employee e1 = new Employee();
e1.name = "Ali";
Employee e2 = new Employee();
e2.name = "Muhammad";


Departement dept = new Departement();
dept.employees = Arrays.asList(e1, e2);

Result result = new Result();
result.departement = dept;

return result;
}

static Result simulateWsResult2() {
Employee e1 = new Employee();
Employee e2 = new Employee();

Departement dept = new Departement();
dept.employees = Arrays.asList(e1, e2);

Result result = new Result();
result.departement = dept;

return result;
}

static Result simulateWsResult3() {
Departement dept = new Departement();

Result result = new Result();
result.departement = dept;

return result;
}

static Result simulateWsResult4() {
Result result = new Result();

return result;
}

Then, the main method looks like:

       public static void main(String[] args) {

System.out.println(getEmployeeNames(simulateWsResult1()));
System.out.println(getEmployeeNames(simulateWsResult2()));
System.out.println(getEmployeeNames(simulateWsResult3()));
System.out.println(getEmployeeNames(simulateWsResult4()));
}

And here's the parsing method: getEmployeeNames

       static List<String> getEmployeeNames(Result r){

return
Optional.ofNullable(r)
.map(result -> result.departement)
.map(dept -> dept.employees)
.map(empList -> empList.stream())
.map(empStream -> empStream.filter(e -> e.name != null).map(e -> e.name))
.orElse(Stream.empty())
.collect(Collectors.toList());
}

Complete code here.

Thanks.

30 September 2016

Gradle is awsome

Gradle the build tool for Java and Android (and other languages) is really powerful.

I am brand new to gradle, this is the first time I try it, and I find it really powerful.

I am working - for fun - on some spring boot + angular project , I decided to go with gradle because I like Groovy (The language that grails is written as a DSL; however gradle is written in Java as well as groovy).

However Eclipse support is not as much good as Maven, but I started the spring boot using gradle as the build tool and every thing is fine.

Now I need to build the spring project as well as the angular project (which is based on angular-seed that uses npm and bower).

I find a plugin for gradle for this task (gradle-node-plugin), this plugin allow you to do:

gradle npm-install

so, the npm install command will run and that what I need, but I need it to run with gradle build command.

First the npm-install task of gradle run by default  package.js file found in src/main/java, So I have to write my own task to make it uses spring-boot conventions (src/main/resources/static)

And thanks to the author of this plugin, he makes it easily to extend his plugin, so I wrote:

task my_npm_install(type: NpmTask) {
description = "Installs dependencies from package.json"
workingDir = file("${project.projectDir}/src/main/resources/static")
args = ['install']
}

Here I am defining a new task (enhanced task), that is of type NpmTask (defined in gradle-node-plugin) then set some properties (defined by the parent task (NpmTask) ), so it can do the job.

so, now I can run: gradle my_npm_task and gradle now will run npm install against the correct package.json file.

What is remaining is to have the this task run after I run gradle build (the build task).

Thanks to the amazing tasks dependency feature provided by gradle, I can refer to some task (provided by java plugin - I think) and make it depends on another task (the one I wrote).

Like this: build.dependsOn(my_npm_install)

Then when I run gradle build, here's the output:

.....
......

So that, the build task will run my_npm_install: (here's the output of gradle build command):

:check
:nodeSetup SKIPPED
:my_npm_install
:build
.......
......

gradle run the my_npm_install task before the build task.

Again, gradle is much more powerful and flexible than maven, and have a better syntax as well.
 

13 September 2016

Rename tv series movies files and subtitles to match

I wrote a simple script to rename the tv series files and its subtitles and remove all extract characters but the episode number along with the extension so the media players (such as VLC) can assign the subtitles automatically.

https://gist.github.com/MuhammadHewedy/a23225acc389e25d393df4659123b880

05 September 2016

shell script to import certificates into java cacerts

I am not the original author, I just some small enhancements



 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
#! /bin/bash

if [ $# -eq 0 ]; then
    echo -e "usage: $0 <host>\nexample: $0 abc.com"
    exit -1
fi

KEYTOOL=../../bin/keytool
HOST=$1
PORT=443
KEYSTOREFILE=cacerts
KEYSTOREFILE_BKUP=$KEYSTOREFILE.`date '+%Y%m%d%H%M'`.'original'
KEYSTOREPASS=changeit

if [ ! -f $KEYSTOREFILE ]; then
    echo -e "You must run this script from the directory jdk/jre/lib/security"
    exit -1
fi

#backup the cacerts file
echo -e "\n\n**** BAKCING UP THE $KEYSTOREFILE TO $KEYSTOREFILE_BKUP ****\n\n"
cp $KEYSTOREFILE $KEYSTOREFILE_BKUP


# get the SSL certificate
echo -e "\n\n**** SAVING THE CERTIFCATE TO ${HOST}.cert ****\n\n"
openssl s_client -connect ${HOST}:${PORT} </dev/null \
    | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > ${HOST}.cert

echo -e "\n\n**** USING keytool AT $KEYTOOL ****\n\n"

# create a keystore and import certificate
echo -e "\n\n**** IMPORTING THE CERTIFICATE... ****\n\n"
"$KEYTOOL" -import -noprompt -trustcacerts \
    -alias ${HOST} -file ${HOST}.cert \
    -keystore ${KEYSTOREFILE} -storepass ${KEYSTOREPASS}

echo -e "\n\n**** PRINTING THE CERTIFICATE AFTER IMPORTED ... ****\n\n"
# verify we've got it.
"$KEYTOOL" -list -v -keystore ${KEYSTOREFILE} -storepass ${KEYSTOREPASS} -alias ${HOST} | grep --color=always $HOST

Using @Transactional to trigger rollback on runtime exceptions

In the last post, I've discussed some thoughts about trying to understand spring transactions.

Although not complete (and it appears it has issues as well). but today's topic is so compact and clear.

Spring follow's EJB transaction on that, it rollback on RuntimeException.

Suppose we have the following method:

@Transactionl
public void myServiceMethod(){
    updateDatabase();
    callSomeQueueOrWebService();
}

so, if the the second method call fails with RuntimeException, then the whole transaction will rollback causing the first method to rolled back.

But if you don't annotate the method with @Transactional, then both method calls (updateDatabase() and callSomeQueueOrWebService() ) will run independently and one might success however the other case fail.

Code:

Service Layer:


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
private String fName = "Mohammad", lName = "Hewedy";

 @Transactional
 public void updateDbAndCallWSTrans() {
  userRepo.save(User.of(null, fName, lName));
  callWsThatThrowsException();
 }

 public void updateDbAndCallWSNoTrans() {
  userRepo.save(User.of(null, fName, lName));
  callWsThatThrowsException();
 }

 public Stream<User> getDbUpdate() {
  return userRepo.findByFNameAndLName(fName, lName);
 }

 private void callWsThatThrowsException() {
  // automatic rollback for runtime exceptions only!
  throw new RuntimeException("EXCEPTION from WS");
 }


The controller layer:


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
 
 @RequestMapping("/tx/user")
 public ResponseEntity<?> getDbUpdate() {
  return ResponseEntity.ok(userService.getDbUpdate().toArray(User[]::new));
 }

 @RequestMapping("/tx/add-w")
 public ResponseEntity<?> updateDbAndCallWSTrans() {
  userService.updateDbAndCallWSTrans();
  return ResponseEntity.ok().build();
 }

 @RequestMapping("/tx/add-wo")
 public ResponseEntity<?> updateDbAndCallWSNoTrans() {
  userService.updateDbAndCallWSNoTrans();
  return ResponseEntity.ok().build();
 }


Note, the call to /tx/add-w will not insert any data into the database, and calls to /tx/user will return empty.

However, the call to /tx/add-wo, will insert the user object into the database, and call to /tx/user will return the inserted users object.

04 September 2016

My thoughts about Spring Transactions

I've some thoughts about spring (and spring-boot) transactions I want to list somewhere, and I think here's a nice place.

Preface

Spring do transaction management through PlatformTransactionManager interface.

This interface has implementation for different kinds of datasources, for example for DataSourceTransactionManagerHibernateTransactionManagerHibernateTransactionManagerJmsTransactionManagerJpaTransactionManager and others.

In a typical JPA application, the JpaTransactionManager is being used.

In SpringBoot, All the JpaRepository are annotated with @Transactional annotation.

This annotation used by spring to do the Declarative transaction management.

So, the Developer uses this annotation to control many things, included the transaction propagation (REQUIRED, REQUIRED_NEW, SUPPORTED, etc), the isolation level (READ_COMMITED, READ_UNCOMMITED, etc.) and the rollback exception, timeout and more.

And behind the since, the platform-specific transaction manager along with the databasource do the actual work.

@Transactionl

So, many people says, use the @Transactional annotation on the service layer, you might need to use it on DAO (now called Repository) layer, but do not use it on the controller layer.

In spring boot, as I said, all the out-of-the-box methods that you get by implementing the JpaRepository are already have the @Transactional annotation (see SimpleJpaRepository). And I think hence the @Transactional on the implementation class itself, then all your repo methods might have the annotation as well.

But this is not the problem, the problem in the Service layer.

@Transactionl in the Service layer

What if I didn't put the Transactional annotation on my service methods?
Then your method will not run in a transaction, so If you have a service method that do the following

public void serviceMethod(){
     insert1();
     insert2();
}

insert1() might succeed and inserts data into the database, while insert2() fails.

In this case you should put the @Transactional annotation on your service method if you want to have all or none.

@Transactionl and noRollbackFor

The rollback is controlled by RuleBasedTransactionAttribute
see http://docs.spring.io/autorepo/docs/spring/current/spring-framework-reference/html/transaction.html#transaction-declarative-rolling-back

In its default configuration, the Spring Framework’s transaction infrastructure code only marks a transaction for rollback in the case of runtime, unchecked exceptions; that is, when the thrown exception is an instance or subclass of RuntimeException. ( Errors will also - by default - result in a rollback). Checked exceptions that are thrown from a transactional method do not result in rollback in the default configuration.
Also,
When the Spring Framework’s transaction infrastructure catches an exception and is consults configured rollback rules to determine whether to mark the transaction for rollback, the strongest matching rule wins 

For rollbackFor and noRollbackFor;

I think spring take the exception thrown and check to see how it is close (in inheritance hierarchy) to the declared exceptions (in rollbackFor and noRollbackFor). If it is closer to an exception declared in rollbackFor more than one declared in noRollbackFor, then the transaction will be rollbacked. and vice versa.

ex:
We have this heirarcy:
class Exception1{}
class Exception2 extends 1{}
class Exception3 extends 2{}

@Transactional(rollbackFor=Exception1.class, noRollbackFor=Exception2 .class)
public void myMethod(){
        throw new Exception3 ();
}

In this case, the transaction will NOT be rollbacked, because the thrown exception (Exception3) is closest to exception in noRollbackFor (Exception2) than the exception in rollbackFor (Exception1)

And if the closest exception declared in rollbackFor, the spring will rollback the tx, otherwise it will not rollback it:

in RuleBasedTransactionAttribute: return !(winner instanceof NoRollbackRuleAttribute);

@Transactionl and lazy evaluation

lazy evaluation is important aspect in web development, it facilities things, lazy evaluation includes not only evaluate lazy associations in the Controller layer, but also evaluation lazy Streams. (Thanks to OpenEntityManagerInViewFilter).

But if you choose to annotate your service method with @Transactional, lazy transaction data will not be available in the Controller layer (and exceptions will be thrown as well).

Here's an example issue.

So, some people says, at service layer get all the data you want and then send it to the controller layer as eager so you don't need the service layer's transaction anymore.

I think for finder services, you might not need to have the @Transactional annotation, so you can make use of the OpenEntityManagerInViewFilter (which is provided by springboot by default) and be able to  enjoy your laziness in the controller layer.(It seems the "Open EntityManager in View" has nothing to do here, it only allow the transaction manager to use an EntityManager created by him. because if you have @Transactional and have this pattern on, the exception will continue to throw if you try to access lazy data out of the transactions.

But in case of services that updates the database, you do need to use the @Transactional annotation to control the things not to go messy!

@Transactionl on controller and service layer

If you have a finder method that returns a lazy object (java 8 stream or lazy association as in hibernate), the if you have the @Transactional annotation on the service layer, then the lazy parts of  object will not be available to the controller layer.

However, if you put the @Transactionl annotation on the controller layer as well (which many people don't recommend) then the lazy part will be available to the controller layer unless you make the transaction propagation on the service layer as REQUIRED_NEW, in this case the service layer will suspend the controller's transaction and start/end it's own transaction and will send back the object to the controller out of its transaction and the problem will still exists.

Conclusion

For now, I'll  - mostly, because some times I do not need it - use @Transactionl on Service methods that do update (Add/update/delete) the database.

However for finder service methods (that query the database), it is better to have the @Transational(readOnly=true) on it, but since I didn't understand - till now- much more, I'll choose not to use this annotation so I can deal with my laziness objects in the controller layer)

Interesting classes to see the source code of:
1. TransactionInterceptor: class source code to see what is happening internally. (very interesting)
2. TransactionAspectSupport:
  a. method: invokeWithinTransaction
  b. method: completeTransactionAfterThrowing (it shows
3. AbstractPlateformTransactoinManager
 a. method: commit (it shows the rollback if done if "setRollbackOnly" method is being called

The post above is based on my thoughts and is might not be valid.

31 August 2016

callback on thread completion

The code illustrate how callback is being accomplished on Thread completion:


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
class Executor {

  public <T> void execute(Callable<T> task, BiConsumer<T, ? super Exception> callback) {
     CallableThread<T> thread = new CallableThread<>(task, callback);
     thread.start();
  }

  class CallableThread<T> extends Thread {

     Callable<T> task;
     BiConsumer<T, ? super Exception> callback;

     public CallableThread(Callable<T> task, BiConsumer<T, ? super Exception> callback) {
        this.task = task;
        this.callback = callback;
     }

     public void run() {
        System.out.println("running task on thread : " + Thread.currentThread().getName());
        try {
            T t = task.call();
            callback.accept(t, null);
        } catch (Exception ex) {
            callback.accept(null, ex);
        }
     }
   }
 }

Caller:
1
2
3
4
5
6
7
8
System.out.println("running task on thread : " + Thread.currentThread().getName());

new Executor().execute(() -> "HELLO WORKD", (result, ex) -> {
    System.out.println("result: " + result);
    System.out.println("exception: " + ex);
});

System.out.println("finished running task on thread : " + Thread.currentThread().getName());


Output:
1
2
3
4
5
running task on thread : main
finished running task on thread : main
running task on thread : Thread-0
result: HELLO WORKD
exception: null

It appears that, the Executor is submitting the Callable to execution and on complete it invokes the callback functional interface.