EXAMINERS AMENDMENT
DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicants
2. This communication is in response to the application filled on 02/15/2024.
3. Claims 1-7, 13, 14, 23, 24, 28, and 29 are pending.
4. Limitations appearing inside {} are intended to indicate the limitations not taught by said prior art(s)/combinations.
Information Disclosure Statement
5. The information disclosure statement (IDS) submitted on 02/15/2024 has been considered by the examiner.
Election/Restrictions
6. Claims 8-12, 15-18, 19-22, and 25-27 are withdrawn from further consideration pursuant to 37 CFR 1.142(b) as being drawn to a nonelected inventions, there being no allowable generic or linking claim. Election was made without traverse in the reply filed on 01/07/2026.
Claim Rejections - 35 USC § 103
7. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
8. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
9. Claims 1-2, 4-6, 13-14, and 24 are rejected under 35 U.S.C. 103 as being unpatentable over “Saliency enabled compression in JPEG framework” to Rahul et al. (hereinafter Rahul) and further in view of “Prioritizing Region of Interest Coding in JPEG2000” to Sanchez et al. (hereinafter Sanchez).
10. Regarding Claim 1, Rahul discloses a computer-implemented method, comprising ([pg. 1142, Abstract, par. 1, ln. 1-4] “Through this paper, a novel region-of-interest (ROI) dependent quantization method in JPEG framework is proposed. The proposed method judiciously quantizes DCT coefficients belonging to salient and non-salient regions of the image. In this work, multiple ROIs are optimally identified and ranked by using variances”):
receiving, via a computing device ([pg. 1142, col. 1, 1 Introduction, par. 1, ln. 1-7] “Usage of image data through the Internet has exponentially increased among the users [1]. Compression is essentially required to manage this high data rate of images without degrading the quality to an unacceptable level. The necessity of accessing the high definition images with quality as of paramount importance has become the major issue in designing such algorithms to operate in real time.”), a plurality of bytes of an encoded image, wherein the encoded image comprises a salient portion ([pg. 1143, Fig. 1, Fig. 2], [pg. 1143, col. 1, par. 2, ln. 1-29] “The work done in multi-level saliency-based compression techniques [15, 21, 26, 27] exhibits an improved trade-off between CR and perceptual quality than using only two-level saliency… JPEG 2000 standard [26] incorporates both two-level and multi-level models of ROI encoding using maximum shift or MAXSHIFT and general scaling based method, respectively. The major challenge in multi-level saliency-based compression technique is the requirement of sending the overhead for the shape of salient regions and their ranks used to grade the saliency. The overhead is proportional to the complexity of the shape and the number of ranks. This is because the complex shaped regions will require a larger number of model parameters which will increase the overhead. Also, the overhead for ranks information will increase with an increase in the number of salient regions, i.e. for the R number of ranks, ⌈log2R⌉ bits per rank will be needed. If the ROI mask is generated for an arbitrary shaped ROI, the decoder needs to reproduce the ROI mask [26], making the decoder computationally complex and increased memory requirement on the decoder side. To reduce the requirement of the overhead information and to make the decoder simple, the ROI shape is approximated as a rectangular box [15, 26], as shown in Fig. 1. The coordinates of the opposite vertices of the rectangular boxes and their rank information are sent to the decoder. This approach saves the overhead information to a good extent but the CR is compromised, as the actual ROI has been approximated by a rectangular bounding box”, [pg. 1144, 3 Proposed method, par. 1, ln. 1 to pg. 1145 to col. 2, 3.1 Number of regions identification and multiple saliency identification, par. 5, ln. 5] “The encoder processes a given image through two paths. The first path generates multi-level saliency map for the input image. The optimal number of classes is adaptively calculated by using efficiency of segmentation [28]. The image is then segmented into the same number of salient classes by maximising between-class variances [29]. Every class is then given a rank based on its importance by using their weighted variances. The class having high weighted variance is given higher rank, i.e. more importance, and vice versa. The second path is used for the adaptive quantisation. The image is decomposed into blocks of size 8 × 8 and each block is ranked based on the input saliency map obtained from the first path by applying probability bound. 2D-DCT coefficients of each block are quantised adaptively by the quantisation parameters modified by the rank of the block. The higher rank block (i.e. more salient) will be lightly quantised and vice versa. The quantised coefficients are then entropy coded and the overhead for the rank of the blocks is reduced by using delta encoding method [30]… Unlike method in [26], where decoder requires reproducing the ROI mask, making the decoder complex, the decoder of our method is simple as ROI information is sent to the decoder by the encoder. The reconstruction of the image is the inverse of the encoding steps. The detailed description of the key steps in the encoding process is given as follows… Salient regions are identified by segmenting the image into Knumber of classes, to be discussed later, by maximising between-class variances. For segmenting the image, the Otsu's segmentation method [28, 29, 31] is extended for K classes. Let these K classes be arbitrary bounded by
K
+
1
intensity levels
(
t
0
,
t
1
,
t
2
,
…
,
t
K
)
as
t
0
<
t
1
<
t
2
<
…
<
t
K
-
1
<
t
k
. For an image with
L
intensity levels,
t
i
(
0
≤
i
≤
K
)
is intensity value of pixels with
t
0
=
0
, and
t
K
=
L
-
1
. Let
i
th class
(
1
<
i
<
K
-
1
)
consist of all the pixels with intensities in the range
[
t
i
-
1
,
t
i
-
1
]
. Whereas, the Kth class consists of pixels with intensity values in the range of
[
t
K
-
1
,
t
K
]
. With theses initial assumptions, probability of the ith class occurrence
(
ω
i
)
, and the class mean
(
μ
i
)
are obtained das follows
ω
i
=
∑
j
=
t
i
-
1
t
i
p
j
,
μ
i
=
1
ω
i
∑
j
=
t
i
-
1
t
i
j
p
j
,
μ
T
=
∑
i
=
1
K
ω
i
μ
i
(4). Here
p
j
is the probability of the pixels with intensity value j, and
(
μ
T
)
is the mean of the image. Thereafter between-class variance
(
σ
K
2
)
can be obtained using the following equation:
σ
K
2
=
∑
i
=
1
K
ω
i
(
μ
i
-
μ
T
)
2
(5)
σ
K
2
is the function of
ω
i
and
μ
i
and these parameters, in turn, are functions of the chosen class boundaries
t
1
,
t
2
,
…
,
t
K
-
1
. It is desired to obtain an optimal set of the class boundaries that result into a maximum value of
σ
K
2
. This can be obtained by iteratively solving (5) for possible Boundary values given in (4). The maximum value of
σ
K
2
is called maximum between-class variance. To identify the total number of classes (K), it is proposed to first obtain the goodness-of-segmentation (GOS)
(
η
K
)
, given in (6). Between-class variance,
σ
K
2
, given in (5), and weighted variance,
S
i
, given in (8), are used to calculate the total variance
σ
T
2
and
η
K
, for initial value of K=2. Value of K is incremented till the inequality given in (7) is satisfied for required value of
(
η
r
)
, typically in the range of 0.8-0.99
σ
T
2
=
σ
K
2
+
∑
i
=
1
K
S
i
,
η
K
=
σ
K
2
σ
T
2
(6) As it can be observed from (6),
η
K
will be <1
η
K
≥
η
r
(
0
≤
η
r
≤
1
)
(7) Choosing number of classes (K) based on GOS helps to avoid the over-segmentation and under-segmentation situations…”);
determining a bounding region for the encoded image, wherein the bounding region is indicative of a location of the salient portion in the encoded image ([pg. 1143, Fig. 1, Fig. 2], [pg. 1143, col. 1, par. 2, ln. 1-29], [pg. 1145, Fig. 5], [pg. 1145, col. 1, 3.3 Block ranking, par. 2, ln. 1 to col. 2, par. 2, ln. 3] “After classifying pixels based on the threshold values
t
1
,
t
2
,
…
,
t
K
-
1
and ranking them according to sorted series of
S
i
, given in (8), we propose to use probability mass function to rank every
8
×
8
blocks used in JPEG. Blocks having pixels with more than one rank may be the one that is on the border, or sometimes on the edges. The block is assigned rank r, whenever the empirically proposed probability bound (10) is satisfied, starting with r=1
∑
i
=
1
r
p
i
≥
1
K
-
r
+
1
,
r
=
1,2
,
…
,
K
(10) where
p
i
is the probability of the ith ranked pixels in the block. To illustrate the probability bound and ranking of blocks, let us assume K=4 and apply (10) on four different blocks of size
3
×
3
shown in Figs. 6a-d. Considering Fig. 6a, for example, it is found that
p
1
=
0.33
,
p
2
=
p
3
=
0
, and
p
4
=
0.66
. The probability bound (10) is then applied, starting with
r
=
1
. The probability bound is satisfied for
r
=
1
, resulting block rank to be 1. Similarly, blocks in Fig. 6b-d get ranks 2, 3, and 4 respectively… DCT coefficients of every ranked block of size
8
×
8
are adaptively quantised as per their importance, estimated in terms of their rank values (r). The quantisation table
(
T
50
)
used in JPEG baseline [10] is proposed to be scaled by a factor
F
r
for rth ranked block
(
1
≤
r
≤
K
)
, and the same is controlled by two variables
V
a
r
and
Q
a
m
given as follows:
F
r
=
V
a
r
+
(
r
-
1
)
Q
a
m
(11)
F
r
=
V
a
r
for
r
=
1
, i.e. for the most salient blocks and value of
F
r
increases by
(
r
-
1
)
Q
a
m
as the saliency of the block decreases (i.e. r increases)…”, [pg. 1147, Fig. 10], [pg. 1147, col. 1, par. 1, ln. 1-10] “Fig. 10 shows the comparison in terms of accuracy of reconstructed ROI at the decoder side, between the proposed method of sending the ROI and the rectangular approximation used in state-of-art saliency enabled methods [15, 26]. The reference images can be seen in Fig. 1. It is observed that the proposed approach of sending ROI information to the decoder by using block ranks retains the ROI structure better than the rectangular approximation of ROI. The average overhead found to be0.00038 bpp while using the rectangular approximation, and 0.0091 bpp while using the proposed method”); and
{progressively} rendering a decoded version of the encoded image, wherein the progressively rendering comprises rendering a high-resolution version of the bounding region, and a low resolution version of a portion outside the bounding region ([pg. 1145, Fig. 5], [pg. 1144, 3 Proposed method, par. 1, ln. 1 to pg. 1145 to col. 2, 3.1 Number of regions identification and multiple saliency identification, par. 5, ln. 5], [pg. 1145, col. 1, 3.3 Block ranking, par. 2, ln. 1 to col. 2, par. 2, ln. 3], [pg. 1146, col. 2, par. 1, ln. 2-16] “When any DWT or DCT methods in [6, 10–12, 14] are applied on the most important regions (r = 1), the mean-square error (MSE) of these regions is higher than overall MSE. The reason for this is while applying these transform-based methods in a region with high variance, the energy compaction is lesser compared to a region with lower variance [18], which results in higher MSE after quantisation to achieve lower bit-rate. A similar example can be referred from rate-distortion curve in Fig. 7, where the PSNR after applying JPEG on the most important regions of the image is always lower than the overall image. This information suggests that the PSNR values provided in Table 3 for the methods in [6, 11, 12, 14], which is for the whole image will have a lower value of PSNR for the regions with (r = 1). It is clear that the proposed method outperforms those reported in [6, 10–12,14]”, [pg. 1147, Table 2, see
R
1
final column]).
One of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Rahul specifically ranks the saliency regions and quantizes them to a lesser extent the lower the rank they have (e.g., region ranked r=1 is most salient and quantized/encoded to a lesser extent then region ranked r=2), and therefore, Rahul discloses rendering a high-resolution version of the bounding region, and a low resolution version of a portion outside the bounding region. With regard to “receiving, via a computing device” one of ordinary skill in the art, before the effective filling date of the claimed invention, would specifically recognize that the encoding and decoding method of Rahul is intended for transmission of images across the internet and/or between computing devices (e.g., server to client, downloading an image online, etc.), and as such, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, that the image would be received, via a computing device. However, Rahul does not specifically disclose progressively rendering an image.
However, Sanchez specifically discloses progressively rendering a decoded version of the encoded image, wherein the progressively rendering comprises rendering a high-resolution version of the bounding region, and a low resolution version of a portion outside the bounding region ([pg. 3, Fig. 3, see ROI in column 1, progressive rendering across columns 2-4], [pg. 4, Fig. 4, see specifically far right column with Face ROI and progressive rendering], [pg. 2, col. 1, 3.1 Single ROI coding, par. 1, ln. 1 to pg. 3, col. 2, par. 1, ln. 11] “The proposed method prioritizes each packet according to its distance from the ROI… This particular prioritization method allows gradual increase in peripheral quality loss during network congestion. Here, we extend the idea of prioritizing data to the JPEG2000 framework to improve the overall visual quality of the decoded image while retaining the ROI coding property. Packets within a ROI receive the highest priority, whereas the surrounding packets receive a priority inversely proportional to their distance measured from the center of the ROI… By using a Gaussian priority distribution, the method assigns a higher priority to the ROI packets and decreases this level of priority smoothly as a function of the distance measured from the center of the ROI. The higher the shape parameter R, the flatter the priority distribution, resulting in more background information being transmitted along with the ROI packets… . Let us assume that N different ROIs need to be encoded with different priorities. The set of N priorities can be expressed as:
H
=
{
P
R
O
I
1
,
P
R
O
I
2
,
P
R
O
I
3
,
…
,
P
R
O
I
N
-
1
,
P
R
O
I
N
}
(4) where
P
R
O
I
n
is the priority assigned to the nth ROI. The proposed method, as described in Section 3.1, may then be separately performed for each priority
P
R
O
I
in H. As a result, each packet is assigned N different priority levels according to Eq. 1. The set of priority levels assigned to packet m in layer l may be expressed as:
h
m
,
l
=
{
p
m
,
l
1
(
r
)
,
p
m
,
l
2
(
r
)
,
p
m
,
l
3
(
r
)
,
…
,
p
m
,
l
N
-
1
(
r
)
,
p
m
,
l
N
(
r
)
}
(5) The final priority level
p
m
,
l
0
(
r
)
of packed m in layer l is then defined as the maximum priority level in
h
m
,
l
:
p
m
,
l
0
=
m
a
x
{
h
m
,
l
}
(6) The main advantage of the proposed method is that it can encode multiple ROIs and include background information along with them. The ROIs may be encoded even after the image has been compressed, provided an equal number of precincts are used in every subband. It is possible for each ROI to have its own priority level in the final code-stream. According to the value of the parameter R, some background information may be included in the layers containing the ROIs. This results in a better visual quality compared to the MAXSHIFT method, where the ROI has to be completely decoded before any background coefficients.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would specifically recognize that Rahul and Sanchez are within the same field of encoding, decoding, and rendering of JPEG images, and as analogous to the claimed invention. The motivation to combine is disclosed in Sanchez, wherein the progressive rendering allows for higher image quality of the rendered image regions of interest during network congestion and allows for the rendering of the entire image as opposed to non-progressive rendering ([pg. 2, col. 1, 3.1 Single ROI coding, par. 1, ln. 1 to pg. 3, col. 2, par. 1, ln. 11], [pg. 4, Fig. 4, see specifically far right column with Face ROI and progressive rendering]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Rahul with the progressive image rendering of Sanchez through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the method of Rahul with the progressive image rendering of Sanchez to obtain the invention as specified in claim 1.
11. Regarding Claim 2, a combination of Rahul and Sanchez teaches the method of claim 1. Rahul further discloses wherein the bounding region is square shaped ([pg. 1143, Fig. 1], [pg. 1147, Fig. 10], [pg. 1145, col. 1, 3.3 Block ranking, par. 2, ln. 1 to col. 2, par. 2, ln. 3]). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that that Rahul discloses dividing the image into bounding regions comprising a multitude of 8x8 squares, or likewise, that a square shape can be used to denote the bounding region with reduced overhead ([pg. 1147, col. 1, par. 1, ln. 1-10]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the method of Rahul with the progressive image rendering of Sanchez to obtain the invention as specified in claim 2.
12. Regarding Claim 4, a combination of Rahul and Sanchez teaches the method of claim 1. Rahul further discloses wherein the salient portion is identified during encoding of the image ([pg. 1143, Fig. 1, Fig. 2], [pg. 1143, col. 1, par. 2, ln. 1-29] [pg. 1144, 3 Proposed method, par. 1, ln. 1 to pg. 1145 to col. 2, 3.1 Number of regions identification and multiple saliency identification, par. 5, ln. 5]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the method of Rahul with the progressive image rendering of Sanchez to obtain the invention as specified in claim 4.
13. Regarding Claim 5, a combination of Rahul and Sanchez teaches the method of claim 2. Rahul further discloses wherein the encoded image is partitioned into a plurality of sequentially ordered regions ([pg. 1144, 3 Proposed method, par. 1, ln. 1 to pg. 1145 to col. 2, 3.1 Number of regions identification and multiple saliency identification, par. 5, ln. 5], [pg. 1144, col. 2, 3.2 Saliency ranking, par. 1, ln. 1 to pg. 1145, col. 1, par. 1, ln. 9] “In order to rank each class, consisting of randomly distributed pixels, weighted variance of the pixels are obtained, using the following equation:
S
i
=
ω
i
σ
i
2
,
i
=
1,2
,
…
,
K
(8) where
σ
i
2
is the ith class, as given in the following equation:
σ
i
2
=
1
ω
i
∑
j
=
t
i
-
1
t
i
(
j
-
μ
i
)
2
p
j
(9)
S
i
for
1
≤
i
≤
K
is sorted in descending order. The ith class pixels will get rank q where q is the position of the sorted
S
i
. Highest weighted variance referrers to the most salient class and gets the highest rank and vice versa, i.e. pixels corresponding to max(
S
i
) will get (r=1) and min(
S
i
) will be ranked (r=K). The main aim is to give more importance to the class with considerable area having high variance. As expressed in (8), a class with a high variance but very small area (small
ω
i
) may get less importance than a class with a relatively lower variance but larger area.”), wherein a lower position in the sequential ordering is indicative of the salient portion of the image ([pg. 1144, 3 Proposed method, par. 1, ln. 1 to pg. 1145 to col. 2, 3.1 Number of regions identification and multiple saliency identification, par. 5, ln. 5], [pg. 1144, col. 2, 3.2 Saliency ranking, par. 1, ln. 1 to pg. 1145, col. 1, par. 1, ln. 9], [pg. 1145, col. 1, 3.3 Block ranking, par. 2, ln. 1 to col. 2, par. 2, ln. 3]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the method of Rahul with the progressive image rendering of Sanchez to obtain the invention as specified in claim 5.
14. Regarding Claim 6, a combination of Rahul and Sanchez teaches the method of claim 1. Rahul further discloses wherein the salient portion is identified at the time the image is received ([pg. 1145, Fig. 5, see Original Image and Path 1 performed prior to encoding]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would specifically recognize that to identify salient portions of the image as disclosed in Rahul are determined prior to encoding but after having received an image, and therefore, it would have been obvious to one of ordinary skill in the art to identify the salient portion at the time the image is received. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the method of Rahul with the progressive image rendering of Sanchez to obtain the invention as specified in claim 6.
15. Regarding Claim 13, a combination of Rahul and Sanchez teaches the method of claim 1. Rahul further discloses wherein the salient portion is a first salient portion, wherein the image comprises a second salient portion, and wherein the determining of the bounding region comprises determining the bounding region so as to prioritize rendering of the first salient portion and the second salient portion ([pg. 1147, Fig. 9 and 10], [pg. 1144, 3 Proposed method, par. 1, ln. 1 to pg. 1145 to col. 2, 3.1 Number of regions identification and multiple saliency identification, par. 5, ln. 5], [pg. 1144, col. 2, 3.2 Saliency ranking, par. 1, ln. 1 to pg. 1145, col. 1, par. 1, ln. 9], [pg. 1145, col. 1, 3.3 Block ranking, par. 2, ln. 1 to col. 2, par. 2, ln. 3]). Specifically, one of ordinary skill in the art, before the effective filing date of the claimed invention, would recognize Rahul contains multiple salient portions where the rendering of a first salient portion is given priority when its rank is lower, because it is quantizing to a lesser extent. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the method of Rahul with the progressive image rendering of Sanchez to obtain the invention as specified in claim 13.
16. Regarding Claim 14, a combination of Rahul and Sanchez teaches the method of claim 1. Rahul further discloses wherein the determining of the bounding region comprises selecting, form a plurality of candidate bounding regions, a bounding region that prioritizes rendering of the salient portion ([pg. 1144, 3 Proposed method, par. 1, ln. 1 to pg. 1145 to col. 2, 3.1 Number of regions identification and multiple saliency identification, par. 5, ln. 5], [pg. 1144, col. 2, 3.2 Saliency ranking, par. 1, ln. 1 to pg. 1145, col. 1, par. 1, ln. 9]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the method of Rahul with the progressive image rendering of Sanchez to obtain the invention as specified in claim 14.
17. Regarding Claim 24, a combination of Rahul and Sanchez teaches the method of claim 1. Rahul discloses wherein the {progressively} rendering of the decoded version of the encoded image comprises pixel-wise rendering ([pg. 1144, 3 Proposed method, par. 1, ln. 1 to pg. 1145 to col. 2, 3.1 Number of regions identification and multiple saliency identification, par. 5, ln. 5], [pg. 1145, col. 1, 3.3 Block ranking, par. 2, ln. 1 to col. 2, par. 2, ln. 3] see 8x8 bocks), and the method further comprises rendering a first number of pixels in the portion inside the bounding region, and a second number of pixels in the portion outside of the bounding region, wherein the first number is {greater than} the second number ([pg. 1145, Fig. 10], [pg. 1144, 3 Proposed method, par. 1, ln. 1 to pg. 1145 to col. 2, 3.1 Number of regions identification and multiple saliency identification, par. 5, ln. 5], [pg. 1145, col. 1, 3.3 Block ranking, par. 2, ln. 1 to col. 2, par. 2, ln. 3]). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that Rahul discloses a pixel-wise rendering in 8x8 blocks, and a first and second region comprising a first number and a second number of pixels, but does not specifically disclose that the first number is greater then the second number (i.e., the first region has more pixels then the second region). Likewise, Rahul does not specifically disclose progressively rendering the image.
However, Sanchez teaches to progressively render the image, and wherein the first region contains a first number of pixels greater then the second number of pixels contained in the second region ([pg. 3, Fig. 3, see ROI in column 1, progressive rendering across columns 2-4], [pg. 4, Fig. 4, see specifically far right column with Face ROI and progressive rendering wherein the face ROI has a higher resolution then non-face ROI pixels, see also B and C, wherein the first region (ROI) is rendered with a number of pixels greater than the second region (non-ROI), which is rendered with 0 pixels], [pg. 2, col. 1, 3.1 Single ROI coding, par. 1, ln. 1 to pg. 3, col. 2, par. 1, ln. 11]). The motivation to combine is analogous to claim 1. One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Rahul with the progressive image rendering and higher resolution regions of Sanchez through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Rahul with the progressive image rendering and higher resolution regions of Sanchez to obtain the invention as specified in claim 24.
18. Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over “Saliency enabled compression in JPEG framework” to Rahul, in view of “Prioritizing Region of Interest Coding in JPEG2000” to Sanchez, and further in view of “JPEG XL next-generation image compression architecture and coding tools” to Alakuijala et al. (hereinafter Alakuijala).
19. Regarding Claim 3, a combination of Rahul and Sanchez teaches the method of claim 1. Rahul teaches wherein the encoded image is encoded in a JPEG{-XL} format ([pg. 1142, Abstract, par. 1, ln. 1-4]). Rahul and Sanchez do not specifically disclose wherein the format is JPEG-XL.
However, Alakuijala teaches wherein the encoding can be in JPEG-XL format ([pg. 1, Abstract, par. 1, ln. 1-8] “An update on the JPEG XL standardization effort: JPEG XL is a practical approach focused on scalable web distribution and efficient compression of high-quality images. It will provide various benefits compared to existing image formats: significantly smaller size at equivalent subjective quality; fast, parallelizable decoding and encoding configurations; features such as progressive, lossless, animation, and reversible transcoding of existing JPEG; support for high-quality applications including wide gamut, higher resolution/bit depth/dynamic range, and visually lossless coding. Additionally, a royalty-free baseline is an important goal. The JPEG XL architecture is traditional block-transform coding with upgrades to each component…”). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that Rahul, Sanchez, and Alakuijala are within the same field of encoding, decoding, and rendering of JPEG images, and as analogous to the claimed invention. The motivation to combine is disclosed in Alakuijala, wherein JPEG-XL provides wide gamut, higher resolution/bit depth/dynamic range, and visually lossless coding as compared to standard JPEG ([pg. 1, Abstract, par. 1, ln. 1-8]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Rahul with the progressive image rendering of Sanchez, and further combined the method of the combination of Rahul and Sanchez with the JPEG-XL format of Alakuijala, through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the method of Rahul with the progressive image rendering of Sanchez and the JPEG-XL format of Alakuijala to obtain the invention as specified in claim 3.
20. Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over “Saliency enabled compression in JPEG framework” to Rahul, in view of “Prioritizing Region of Interest Coding in JPEG2000” to Sanchez, and further in view of “Learning to Detect A Salient Object” to Liu et al. (hereinafter Liu).
21. Regarding Claim 7, a combination of Rahul and Sanchez teaches the method of claim 6. Rahul and Sanchez do not specifically disclose wherein the salient portion is identified by applying a trained machine learning model.
However, Liu teaches wherein the salient portion is identified by applying a trained machine learning model ([pg. 1, col. 1, Abstract, par. 1, ln. 1-9] “We study visual attention by detecting a salient object in an input image. We formulate salient object detection as an image segmentation problem, where we separate the salient object from the image background. We propose a set of novel features including multi-scale contrast, center-surround histogram, and color spatial distribution to describe a salient object locally, regionally, and globally. A Conditional Random Field is learned to effectively combine these features for salient object detection.”, [pg. 3, col. 1, CRF for Salient Object Detection, par. 1, ln. 1 to col. 2, par. 2, ln. 7] “We formulate the salient object detection problem as a binary labeling problem by separating the salient object from the background. In the Conditional Random Field (CRF) framework [13], the probability of the label
A
=
{
a
x
}
given the observation image I is directly modeled as a conditional distribution
P
A
I
=
1
z
e
x
p
(
-
E
(
A
|
I
)
)
, where Z is the partition function. To detect a salient object, we define the energy
E
(
A
|
I
)
as a linear combination of a number of K salient features
F
k
(
a
x
,
I
)
and a pairwise feature
S
a
x
,
a
x
'
,
I
:
E
A
I
=
∑
x
∑
k
=
1
K
λ
k
F
k
a
x
,
I
+
∑
x
,
x
'
S
a
x
,
a
x
'
,
I
, (3) where
λ
k
is the weight of the kth feature, and
x
,
x
'
are two adjacent pixels. Compared with Markov Random Field (MRF), one of advantages of CRF is that the feature functions
F
k
(
a
x
,
I
)
and
S
a
x
,
a
x
'
,
I
can use arbitrary low-level or high-level features extracted from the whole image. CRF also provides an elegant framework to combine multiple features with effective learning.”, [pg. 6, col. 1, Effectiveness of features and CRF learning, par. 1, ln. 1 to par. 2, ln. 4] “To evaluate the effectiveness of each salient object feature, we trained four CRFs: three CRFs with individual features and one CRF with all three features. Figure 10 shows the precision, recall, and F-measure of these CRFs on the image sets A and B… Figure 11 shows the feature maps and labeling results of several examples. Each feature has its own strengths and limitations. By combining all features with the pairwise feature, the CRF successfully locates the most salient object.”, [pg. 6, Figure 11, see far right]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Rahul, Sanchez, and Liu as within the same field of image processing related to saliency determination, and Rahul and Liu as specifically related to the field of saliency determination for an image. The motivation to combine is disclosed in Liu, wherein Liu specifically discloses saliency as applicable to image compression ([pg. 1, col. 1, 1. Introduction, par. 1, ln. 4-8] “There are many applications for visual attention, for example, automatic image cropping [23], adaptive image display on small devices [4], image/video compression, advertising design [7], and image collection browsing”), and furthermore, would have been obvious to one of ordinary skill in the art, in that the machine learning model of Liu is far more adaptable than the static method of Rahul since it can learn multiple features on a local, regional, and global image level ([pg. 4, col. 1, 4. Salient Object Features, par. 1, ln. 1 to pg. 5, col. 2, par. 5, ln. 5] “…we introduce local, regional, and global features that define a salient object… Contrast is the most commonly used local feature for attention detection… we simply define the multiscale contrast feature
f
c
x
,
I
as a linear combination of contrasts in the Gaussian image pyramid:
f
c
x
,
I
=
∑
l
=
1
L
∑
x
'
∈
N
(
x
)
I
l
x
-
I
l
(
x
'
)
2
(9)… An example is shown in Figure 5. Multi-scale contrast highlights the high contrast boundaries by giving low scores to the homogenous regions inside the salient object… As shown in Figure 2, the salient object usually has a larger extent than local contrast and can be distinguished from its surrounding context. Therefore, we propose a regional salient feature. Suppose the salient object is enclosed by a rectangle R. We construct a surrounding contour
R
S
with the same area of R, as shown in Figure 6 (a). To measure how distinct the salient object in the rectangle is with respect to its surroundings, we can measure the distance between R and
R
S
using various visual cues such as intensity, color, and texture/texton. In this paper, we use the
χ
2
distance between histograms of RGB color:
χ
2
R
,
R
S
=
1
2
∑
(
R
i
-
R
S
i
)
2
R
i
+
R
S
i
. We use histograms because they are robust global description of appearance. They are insensitive to small changes in size, shape, and viewpoint. Another reason is that the histogram of a rectangle with any location and size can be very quickly computed by means of integral histogram introduced recently [20]. Figure 6 (a) shows that the salient object (the girl) is most distinct using the
χ
2
histogram distance… The center-surround histogram is a regional feature. Is there a global feature related to the salient object? We observe from Figure 2 that the wider a color is distributed in the image, the less possible a salient object contains this color. The global spatial distribution of a specific color can be used to describe the saliency of an object. To describe the spatial-distribution of a specific color, the simplest approach is to compute the spatial variance of the color. First, all colors in the image are represented by Gaussian Mixture Models (GMMs)
w
c
,
μ
c
,
Σ
C
c
=
1
C
, where
w
c
,
μ
c
,
Σ
C
is the weight, the mean color and the covariance matrix of the cth component. Each pixel is assigned to a color component with probability: … (12), Finally, the color spatial-distribution feature
f
s
x
,
I
is defined as a weighted sum:
f
s
x
,
I
∝
∑
c
p
c
I
x
▪
1
-
V
c
.
(15)… Figure 8 (b) shows color spatial-distribution feature maps of several example images. The salient objects are well covered by this global feature… As shown in Figure 8 (c), center-weighted, color spatial variance shows a better prediction of the saliency of each color. To verify the effectiveness of this global feature, we plot the color spatial-variance versus average saliency probability curve on the image set A, as shown in Figure 9. Obviously, the smaller a color variance is, the higher probability the color belongs to the salient object.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Rahul with the progressive image rendering of Sanchez, and further combined the method of the combination of Rahul and Sanchez with the machine learning model for saliency detection of Liu, through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Rahul with the progressive image rendering of Sanchez and the machine learning model for saliency detection of Liu to obtain the invention as specified in claim 7.
22. Claim 23, and 28-29 are rejected under 35 U.S.C. 103 as being unpatentable over “Saliency enabled compression in JPEG framework” to Rahul, in view of “Prioritizing Region of Interest Coding in JPEG2000” to Sanchez, and further in view of U.S. Patent No. 6,314,452 to Dekel et al. (hereinafter Dekel).
23. Regarding Claim 23, a combination of Rahul and Sanchez teaches the method of claim 1. Rahul further discloses wherein the receiving of the plurality of bytes of the encoded image occurs over a communications network ([pg. 1142, col. 1, 1 Introduction, par. 1, ln. 1-7]), and wherein the {progressively} rendering of the decoded version of the encoded image is {based on one or more network} characteristics of the communications network ([pg. 1143, col. 2, par. 1, ln. 2-10] “The required bit-rate after encoding by JPEG baseline is mainly controlled in the quantisation phase. The DCT coefficients of all the blocks are quantised by fixed quantisation parameter of its quantisation table (T, i.e. a matrix of quantisation step sizes). According to JPEG standard, the quantisation table can be configured as per the bit-rate requirement [18]. There have been series of several quantisation tables developed and are widely used for the requirement of higher CR or improved reconstructed image quality.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that the bit rate requirement is typically controlled via the communication network (i.e., number of bits that can be transmitted at any given time is a result of the network throughput, traffic, etc.). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that while Rahul discloses controlling the quantization of the image based on the bit rate requirement, Rahul does not specifically disclose wherein the image is progressively rendered based on said bit rate requirement. Therefore, Rahul fails to disclose progressively rendering based on the one or more network characteristics.
However, Sanchez teaches progressively rendering an image ([pg. 3, Fig. 3, see ROI in column 1, progressive rendering across columns 2-4], [pg. 4, Fig. 4, see specifically far right column with Face ROI and progressive rendering], [pg. 2, col. 1, 3.1 Single ROI coding, par. 1, ln. 1 to pg. 3, col. 2, par. 1, ln. 11]). The motivation to combine remains analogous to claim 1. One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Rahul with the progressive image rendering of Sanchez through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. However, Sanchez fails to specifically discloses wherein said progressive rendering is based on the one or more network characteristics. Therefore, a combination of Rahul and Sanchez fails to discloses wherein the progressive rendering is based on the one or more network characteristics.
However, Dekel teaches wherein the progressive rendering is based on one or more network characteristics ([col. 20, ln. 25-46] “As ROI data is transmitted to the client 110, the rendering algorithm is performed at certain time intervals of a few seconds. At each point in time, only one rendering task is performed for any given displayed image. To ensure that progressive rendering does not become a bottleneck, two rates are measured: the data block transfer rate and the ROI rendering speed. If it predicted that the transfer will be finished before a rendering task, a small delay is inserted, such that rendering will be performed after all the data arrives. Therefore, in a slow network scenario (as the Internet often is), for almost all of the progressive rendering tasks, no delay is inserted. With the arrival of every few kilobytes of data, containing the information of a few data blocks, a rendering task visualizes the ROI at the best possible quality. In such a case the user is aware that the bottleneck of the ROI rendering is the slow network and has the option to accept the current rendering as a good enough approximation of the image and not wait for all the data to arrive.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Rahul, Sanchez, and Dekel as within the same field of encoding, decoding, and rendering of JPEG images, and as analogous to the claimed invention. The motivation to combine the method of the combination of Rahul and Sanchez with the network characteristic-based rendering of Dekel is disclosed in Dekel, wherein it removes bottlenecking as a result of the progressive rendering in the case the network speed is sufficient to render the entire image ([col. 20, ln. 25-46]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Rahul with the progressive image rendering of Sanchez, and further combined the method of the combination of Rahul and Sanchez with the network characteristic-based rendering of Dekel through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Rahul with the progressive image rendering of Sanchez and the network characteristic-based rendering of Dekel to obtain the invention as specified in claim 23.
24. Regarding Claim 28, the claim language is analogous to claim 1, with the exception of “A computing device, comprising: one or more processors; and data storage, wherein the data storage has stored thereon computer-executable instructions, that when executed by one or more processors, cause the computing device to carry out functions comprising:” wherein the remainder of the claim is analogous to claim 1. Rahul and Sanchez do not specifically disclose one or more processors, or a data storage unit, though it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, that Rahul discloses a method intended to be performed on a computer ([pg. 1142, Abstract, par. 1, ln. 1-4], [pg. 1142, col. 1, 1 Introduction, par. 1, ln. 1-7]).
However, Dekel specifically discloses a computing device comprising: one or more processors ([Fig. 1], [col. 4, ln. 3-9] “The client computer 110 and server computer 120 may comprise a PC-type computer operating with a Pentium-class microprocessor, or equivalent. Each of the computers 110 and 120 may include a cache 111, 121 respectively as part of its memory. The server may include a suitable storage device 122, such as a high-capacity disk, CD-ROM, DVD, or the like.”); and data storage, wherein the data storage has stored thereon computer-executable instructions, that when executed by one or more processors, cause the computing device to carry out functions ([Fig. 1], [col. 4, ln. 3-9], [col. 36, ln. 18-28]). The motivation to combine the computing device of Dekel to perform the method of the combination of Rahul and Sanchez is disclosed in Dekel ([col. 4, ln. 11-30] “The client computer 110 and server computer 120 may be connected to each other, and to other computers, through a communication network 130, which may be the Internet, an Intranet (e.g., a local area network), a wide-area network, a wireless network, or the like. Those having ordinary skill in the art will recognize that any of a variety of communication networks may be used to implement the present invention… using any browser type application, the user of the client computer 110 connects to a Web Server 140 or directly to the Imaging server 120 as described in FIG. 1. In step 201, the user then selects, using common browser tools, an image residing on the image file storage device 122. The corresponding URL request is received and processed by the imaging server 120. In the case where the results of previous computations on the image are not present in the imaging cache 121, the server 120 performs a fast preprocessing algorithm (described in further detail later in section 6.1). The result of this computation is inserted into the cache 121.”), and would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, in that it allows for real world application of the method (e.g., across server to client through the internet). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Rahul with the progressive image rendering of Sanchez, and further combined the method of the combination of Rahul and Sanchez with the computing device, processor, and data storage of Dekel through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Rahul with the progressive image rendering of Sanchez and the computing device, processor, and data storage of Dekel to obtain the invention as specified in claim 28.
25. Regarding Claim 29, the claim language is analogous to claim 1, with the exception of “An article of manufacture comprising one or more computer readable media having computer-readable instructions stored thereon that, when executed by one or more processors of a computing device, cause the computing device to carry out functions that comprise:”, wherein the remainder of the claim is analogous to claim 1. Rejections analogous to claim 28 are further applicable to claim 29. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Rahul with the progressive image rendering of Sanchez and the computer readable media of Dekel to obtain the invention as specified in claim 29.
Conclusion
26. The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. See PTO-892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAULO ANDRES GARCIA whose telephone number is (703)756-5493. The examiner can normally be reached Mon-Fri, 8-4:30PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached on (571)272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAULO ANDRES GARCIA/Examiner, Art Unit 2669 /CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669