Showing posts with label encoding. Show all posts
Showing posts with label encoding. Show all posts

19 May 2010

request.getCharacterEncoding() always returns null

Alsalamo Alykom,

If you suffer from this problem, so this is the place you need to visit!

Well, you need to get the character encoding of the request sender in Servlets and you have tried using this method again and again but without any results!

So, this is not the problem of the Servlets, It is the problem of the browser you use!

Yes, try the following code to make sure that the Servlet engine can interpret the character encoding of your request, but only when they are really sent

the following code written suing Commons-http, and need commons-logging and commons-codes:

package com.forat;

import java.io.IOException;

import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpMethod;
import org.apache.commons.httpclient.methods.PostMethod;

public class Client {
public static void main(String[] args) throws IOException{
HttpClient client = new HttpClient();
HttpMethod method = new PostMethod("http://localhost:8080/aServer/AServlet");
method.setRequestHeader("Content-Type", "text/plain; charset=GB2312");
client.executeMethod(method);
}
}


And here's the Servlet code:

package com.forat;

import java.io.IOException;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

/**
* Servlet implementation class AServlet
*/
public class AServlet extends HttpServlet {
private static final long serialVersionUID = 1L;

protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
System.out.println(request.getCharacterEncoding());
}

}



And here's the output:
GB2312


For Firefox, try Firebug to make sure that firefox doesn't send the Content-Type as a request header even if you set.

Here's a sample HTML page that sends to the same above servlet:

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=GB2312">
</head>
<body>

<form action="AServlet" method="post">
<input type="text" name="name">
<input type="submit">
</form>
</body>
</html>


and the result of the servlet is:
null


And here's a screenshot from Firebug displaying all request header but doens't display Content-type.

07 May 2010

Enable Arabic (non-ASCII) characters in request parameteres of "GET" method

Al salamo Alykom,

If you have a JSP page that sends a request with non-ASCII data to a Servlet, here's the steps to enable the Servlet from interrupting the correct character.

First, for your Servlet (Server) to interrupt the character encoding correctly, It needs to know in what encoding the client will send the request.

1- suppose you are sending the following JSP Page:
<%@ page language="java" contentType="text/html; charset=UTF-8"
pageEncoding="UTF-8"%>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Insert title here</title>
</head>
<body>

<form action="SomeServlet" method="get">
Enter your name (in arabic, or in any non-ASCII language)<input type="text" name="username" />
<input type="submit">
</form>

</body>
</html>


2- create the following Class:

package com.forat.web.util;


import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.util.Collections;
import java.util.Enumeration;
import java.util.HashMap;
import java.util.Map;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletRequestWrapper;

/**
* Used to overrides the implementations of
* getParameterXXXXX methods in {@link HttpServletRequest} <br />
* It decoded the query string for the incoming requests Requests before calling the getParameterXXXX methods
*
* @author mhewedy
*/
public class EncodedHttpServletRequest extends HttpServletRequestWrapper {

private String encoding;
private HttpServletRequest request;
private Map paramMap;

/**
* This Constructor is the same as if you create an instance by calling
* EncodedHttpServletRequest(request, "GB2312")
* @param request
* @see #EncodedHttpServletRequest(HttpServletRequest, String)
*/
public EncodedHttpServletRequest(HttpServletRequest request) {
this(request, "UTF-8");
}

/**
* Create instance of this class, trying to set the character encoding - that will be used to decode the query
* string - by calling {@link HttpServletRequest#getCharacterEncoding()}, if its value is null,
* then the <em>encoding</em> parameter's value will be used instead
* @param request
* @param encoding encoding to be used to encode the request's query string
*/
public EncodedHttpServletRequest(HttpServletRequest request, String encoding) {
super(request);
this.request = request;
this.encoding = encoding;
}

public Map getParameterMap() {
if (paramMap == null) {
String queryString = request.getQueryString();
if (queryString == null) return Collections.unmodifiableMap(paramMap = new HashMap());
synchronized (this) {
if (paramMap == null) {
paramMap = new HashMap();
fillParameterMap(queryString);
}
}
}
return Collections.unmodifiableMap(paramMap);
}

private void fillParameterMap(String queryString) {
try {
String[] nvPair = queryString.split("&");
for (String pair : nvPair) {
String[] pairArr = pair.split("=");
String key = pairArr[0];
String value = URLDecoder.decode(pairArr[1], encoding) ;

if (paramMap.containsKey(key)) {
String[] valArr = (String[]) paramMap.get(key);
String[] newValArr = new String[valArr.length + 1];
System.arraycopy(valArr, 0, newValArr, 0, valArr.length);
newValArr[newValArr.length - 1] = value;
paramMap.put(key, newValArr);
}else {
paramMap.put(key, new String[] {value});
}
}
}catch(UnsupportedEncodingException ex) {
throw new RuntimeException(ex.getMessage(), ex);
}catch(Exception ex) {
throw new RuntimeException("Malformed query string: " + queryString, ex);
}
}

public String getParameter(String name) {
Object value = getParameterMap().get(name);
if (value == null)
return null;
return ((String[])getParameterMap().get(name))[0];
}

public String[] getParameterValues(String name) {
Object value = getParameterMap().get(name);
if (value == null)
return null;
return ((String[])getParameterMap().get(name));
}

public Enumeration getParameterNames() {
return Collections.enumeration(getParameterMap().keySet());
}
}


3- create that filter :

package com.forat.web.util;

import java.io.IOException;
import javax.servlet.Filter;
import javax.servlet.FilterChain;
import javax.servlet.FilterConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.http.HttpServletRequest;

/**
* Set the encoding for the POST requests if one not specified by the incoming request
* @author mhewedy
*/
public class EncodingFilter implements Filter {

public void init(FilterConfig fConfig) throws ServletException {}

public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {

String HTTP_METHOD = ((HttpServletRequest) request).getMethod();
if ("GET".equalsIgnoreCase(HTTP_METHOD)) {
// for GET requests
request = new EncodedHttpServletRequest((HttpServletRequest) request, "UTF-8");
}else if ("POST".equalsIgnoreCase(HTTP_METHOD)) {
// for Posted requests
request.setCharacterEncoding("UTF-8");
response.setCharacterEncoding("UTF-8");
}

chain.doFilter(request, response);
}

public void destroy() {}
}



and here's its mapping:

<filter>
<display-name>EncodingFilter</display-name>
<filter-name>EncodingFilter</filter-name>
<filter-class>com.forat.web.util.EncodingFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>EncodingFilter</filter-name>
<url-pattern>/*</url-pattern>
<dispatcher>REQUEST</dispatcher>
<dispatcher>FORWARD</dispatcher>
<dispatcher>INCLUDE</dispatcher>
<dispatcher>ERROR</dispatcher>
</filter-mapping>


4- and here's the Servlet that accepts the User input:

package com.forat.web;

import java.io.IOException;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

/**
* Servlet implementation class SomeServlet
*/
public class SomeServlet extends HttpServlet {
private static final long serialVersionUID = 1L;

/**
* @see HttpServlet#HttpServlet()
*/
public SomeServlet() {
super();
// TODO Auto-generated constructor stub
}

/**
* @see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse response)
*/
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
System.out.println(request.getParameter("username"));
}

}



Good luck :)
(for more comprehensive tutorial, please see http://www.arabteam2000-forum.com/index.php?showtopic=220589 (In Arabic) )

06 May 2010

Beware, your text editor encodes your characters

Al salamo Alykom,

Actually nowadays I am falling in some encoding issue that making me walk thinking in how characters being encoded!

And here's another note, actually I got from BalusC, from stackoverflow

Ok, When you intend to use some character encoding in your html/jsp/php/... page, you have to know that your text editor is actually encoding your text.

What does this mean??

Well, suppose you have a html page that you need to submit in another encoding rather than UTF-8 (say, GB3212 - some chines encoding)

If you write any field value in the text editor (as opposite to make the user enter it through text boxes), you will get the characters corrupted. (this happens when using hidden fields)

For example, this page:

<HTML>
<meta http-equiv='Content-Type' content='text/html; charset=gb2312'>
<BODY >
<form name="form" method="post" action="http://localhost:8080/testChinaEncoding/EncodingServlet" accept-charset="gb2312" accept="gb2312">
<input type="text" name="username" size="50" value="美白祛斑—做完美女人"> <!-- UTF-8 characters -->
<input type="submit">
</form>
</BODY>
</HTML>


The point is, when open this page in a browser, you will notice that the characters appear in the text box, are corrupted!
Yes, this is because you are typing these characters in the text editor using one encoding (usually UTF-8), and want to display these characters using other character encoding (GB2312)!

And here's a screen-shot from the output page:


Note, the big difference from the character being written in the form, and the characters being displayed!

for more info, please see: http://balusc.blogspot.com/2009/05/unicode-how-to-get-characters-right.html

21 April 2010

How to send Query String throw the form GET method using specific encoding

In this post, I'll not talk much about Java, but about URL Encoding in General, although, I'll use java code to Illustrate that.

As you might know, the Get method of the HTTP protocol puts the query string in the request header.

example:

<html>
<head>
</head>
<body>
<form method="GET" action="/some/server">
<input type="text" name="name1" value="val1" />
<input type="text" name="name2" value="val2" />
<input type="text" name="name3" value="val3" />
<input type="submit" />
</form>
</body>
</html>


When you submit this form, the query string will looks like:
name1=val1&name2=val2&name3=val3


this appears for you in the browser navigation box.

for more info see:
http://en.wikipedia.org/wiki/Query_string and http://en.wikipedia.org/wiki/Percent-encoding

As it is clear from wiki links, not all characters can be included as is in the Query String, that characters get "URL Encoded" , and this URL encoding converts these characters in the form %HH, and this conversion (encoding) done based on the encoding schema, as the UTF-8 encoding schema encodes 'أ' to '%D8%A3' whereas other encoding such as ISO-8859-6 encodes it to '%C3'.

BTW, character 'أ', is the first character in the Arabic Language (proudly my native language).


The browser do URL Encoding to the Query String and sends this request to the server, So the server has to "Decode" this Query String back to the original characters to manipulate it as it's needs (inserts in db, etc ..)

Example:

index.html:

<html>
<head>
<meta http-equiv='Content-Type' content='text/html; charset=UTF-8'>
</head>
<body>
<form method="GET" action="TestServlet" accept-charset="UTF-8" >
<input type="text" name="name1" /> <!-- put here any non-ascii char, ex أ >
<input type="submit" />
</form>
</body>
</html>


TestServlet.java

package com.forat;

import java.io.IOException;
import java.net.URLDecoder;
import java.util.Enumeration;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

public class TestServlet extends HttpServlet {
private static final long serialVersionUID = 1L;

/**
* @see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse response)
*/
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
String qString = request.getQueryString();

System.out.println("Raw Query String: " + qString);
System.out.println("Decoded Query String: " + URLDecoder.decode(qString, "UTF-8"));
}
}


Try to change the encoding of index.html and TestServlet.jsp and notice the different encoding representations.

09 January 2009

Enabling Arabic in Java web applications (JSP apps and Struts apps )

Update here:
http://m-hewedy.blogspot.com/2010/05/enable-arabic-non-ascii-characters-for.html


Enabling Arabic (Unicode) in Java web applications (JSP apps and Struts apps )

One of the most big problems you can face as a java web developer is how
to make you web application accepts Arabic (non-ISO-8859-1 characters)
characters from the users in the input text boxes.

In JSP/Servlet applications, a small googling can solve the problem, but
with a framework such as Struts, the matter differ somewhat.

If your web application uses the Database ( the most if not all does )
you should insure that the problem is not in the Database itself that cannot
save Arabic characters, you should try hand-entering Arabic words in varchar
and nvarchar fields, if it accepts Arabic characters, then the problem is in
the JSP/Servlet or Struts and you will find the solution here.

1- To Enable Arabic for request parameteres of "POST" method:

For pure JSP/Servlet Applications :

Two steps and every thing well be done.

First: in each JSP page write this tag at the top of the page :

<%@page language="java" contentType="text/html; charset=UTF-8"%>

Second : in each Servlet that works as the controller for you JSPs, write these two statements at the top of your doPost() :

request.setCharacterEncoding("UTF-8");
response.setCharacterEncoding("UTF-8");

That’s all about Enabling Arabic character acceptation in your JSP forms

For Struts Applications:

Also two steps :
First: in each JSP page write this tag at the top of the page :

<%@page language="java" contentType="text/html; charset=UTF-8"%>

Second: write a Filter class the wraps your org.apache.struts.action.ActionServlet class and put these two statements in its doFilter() method

request.setCharacterEncoding("UTF-8");

response.setCharacterEncoding("UTF-8");

example filter :

public class ArabicEncodingFilter implements Filter {

private void doBeforeProcessing(ServletRequest request, ServletResponse response)

throws IOException, ServletException {

request.setCharacterEncoding("UTF-8");

response.setCharacterEncoding("UTF-8");

}

private void doAfterProcessing(ServletRequest
request, ServletResponse response)
throws IOException, ServletException {

}

public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)throws IOException, ServletException {

doBeforeProcessing(request, response);

Throwable problem = null;

try {

chain.doFilter(request, response);

} catch(Throwable t) {

problem = t;

t.printStackTrace();

}

doAfterProcessing(request, response);

}}}

And then wrap all your Servlets with you Filter by putting this in web.xml :

<filter>

<filter-name>EncodingFilter</filter-name>

<filter-class>ArabicEncodingFilter</filter-class>

</filter>

<filter-mapping>

<filter-name>EncodingFilter</filter-name>

<url-pattern>/*</url-pattern>

</filter-mapping>

2- Enable Arabic for request parameteres of "GET" method:

It depends on your server (tomcat/JBoss, etc), In tomcat 6 do the following:

set the URIEncoding attribute of the element in /conf/server.xml to UTF-8.

Or you can retrieve the request QueryString as is and URLDecode it using the desired encoding (UTF-8).

You can Wrtie a HttpServletRequestWrapper that wraps your HttpServletRequest's getParameter methods .

see :

http://balusc.blogspot.com/2009/05/unicode-how-to-get-characters-right.html
http://java.sun.com/developer/technicalArticles/Intl/HTTPCharset/
http://www.joelonsoftware.com/articles/Unicode.html
http://java.sun.com/javaee/5/docs/tutorial/doc/bnayb.html



for more, complete solustion, see : http://m-hewedy.blogspot.com/2010/05/enable-arabic-non-ascii-characters-for.html