DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendments
2. The Amendment filled 04/29/2026 in response to Non-Final Office Action mailed 02/10/2026 has been entered.
3. Claims 1-7, 13, 14, 23, 24, 28, and 29 are currently pending.
Election/Restrictions
4. Claims 8-12, 15-18, 19-22, and 25-27 are withdrawn from further consideration pursuant to 37 CFR 1.142(b) as being drawn to a nonelected inventions, there being no allowable generic or linking claim. Election was made without traverse in the reply filed on 01/07/2026.
Response to Arguments
5. Applicant’s arguments, see pg. 1-2, with regards to the 103 rejections of claims 1-2, 4-6, 13-14, and 24 have been fully considered and partially persuasive. Specifically, the applicant argues that Sanchez fails to specifically teach partitioning the encoded image into a plurality of sequentially ordered regions including a first region which is associated with the bounding region and is a most salient portion of the encoded image, a plurality of second regions which concentrically surround the first region, and a plurality of third regions which concentrically surround the plurality of second regions, and progressively rendering a decoded version of the encoded image by sequentially rendering the first region, the plurality of second regions, and the plurality of third regions, wherein the first region is rendered with a higher resolution than the plurality of second regions, and the plurality of third regions. Specifically, the applicant contends that the Gaussian distribution used for ordering in Sanchez is not analogous to the sequentially ordered, concentrically surrounded first, second, and third regions, and that they are not rendered sequentially. The examiner disagrees, with the exception of “sequentially rendering”. Specifically, the examiner notes that a Gaussian distribution, based on a distance from the ROI center as disclosed in Sanchez, would be understood by one of ordinary skill in the art to be a series of regions, concentrically surrounding each other, that are ordered based on the inverse of the distance from the center ([pg. 3, Fig. 3, see ROI in column 1, progressive rendering across columns 2-4], [pg. 4, Fig. 4, see specifically far right column with Face ROI and progressive rendering], [pg. 2, col. 1, 3.1 Single ROI coding, par. 1, ln. 1 to pg. 3, col. 2, par. 1, ln. 11]). In this case, it would be apparent that it is effectively analogous to Fig. 3 320, which shows the concentrically surrounded first, second, and third region and so forth, sequentially ordered from 1-9. To substantiate this, the examiner has provided a video (see PTO-892, “Multivariate Gaussian distribution” by @kamperh and associate link, which will be references in timestamps format), which explains and displays what a standard multivariate Gaussian distribution of a 2-D image would effectively look like (see 3:40 to 5:14). Specifically, the examiner notes that if you let
x
2
=
y
distance from the center of an ROI, and
x
1
=
x
distance from the center of an ROI, you would effectively obtain the same distribution as provided in the video at 3:40, since
x
2
=
y
=
0
,
x
1
=
x
=
0
(i.e., the x and y distance to ROI=0) would be the ROI center, and thus the maximal probability on the distribution, and represented by the yellow region in the video. Likewise, a
x
2
=
y
=
1
,
x
1
=
x
=
0
, using the standard distribution (e.g., as provided in 3:40 of the video), would be in the blue/green region of the ROI, which represents a lower probability than the yellow region, or in the case of Sanchez, a lower rank. As such, since Sanchez specifically discloses assigning ranks based on the inverse distance (i.e., the closer to the center, the higher the rank), and using a Gaussian distribution to accomplish this ranking, the achieved affect is a set of regions that would roughly match with the standard distribution as provided in 3:40 of the video, and/or Fig. 3 320 of the claimed invention. Specifically, in relation to Fig. 3 320, you can see the same affect, for example: let 1 be the center region of the ROI as in Fig. 3 320, therefore,
x
2
=
y
=
0
,
x
1
=
x
=
0
, rank = yellow and/or highest rank. The adjacent pixel above 1 would have
x
2
=
y
=
1
,
x
1
=
x
=
0
, therefore, rank = blue/green, in this case the second highest rank. For the pixels
x
2
=
y
=
2
,
x
1
=
x
=
1
, which would be the pixel two steps up and one step to the right of the center, you would get a rank=purple, being a rank below both yellow and blue/green. The examiner specifically notes that this is using a completely unmodified standard Gaussian distribution as an example, and one of ordinary skill in the art would recognize that you can easily modify the distribution to give more weight to one axis distance or the other (e.g., see Gaussian at 4:52, where the
x
2
has a wider distribution than
x
1
). This is specifically accomplished in Sanchez using equation (2) as a method to choose the shape parameter R ([pg. 2, col. 2, par. 3, ln. 1-6, Equation (2)] “The shape parameter R is used to change the rate at which the priority drops. A large Rresults in a flat distribution, while a small R results in a fast decaying distribution. Let the horizontal and vertical size of the image be denoted by H and V, respectively. The diagonal length of the image (measured in pixels) can be calculated as:
S
=
H
2
+
V
2
”). Therefore, Sanchez effectively teaches wherein the partitioning of the encoded image into a plurality of sequentially ordered regions including a first region which is associated with the bounding region and is a most salient portion of the encoded image, a plurality of second regions which concentrically surround the first region, and a plurality of third regions which concentrically surround the plurality of second regions, and progressively rendering a decoded version of the encoded image by sequentially rendering the first region, the plurality of second regions, and the plurality of third regions, wherein the first region is rendered with a higher resolution than the plurality of second regions, and the plurality of third regions. The examiner further notes that with regard to the “sequentially” rendering, the BRI of “sequentially rendering” fails to specifically disclose how “sequentially rendering” should be performed (e.g., is the first region fully rendered before the following regions, or is the sequentially referring to the order by which the packets quality is rendered and subsequently increased, or in the order they are completed). The examiner specifically notes that this contradicts what the examiner stated with regard to Sanchez during the interview dated 04/28/2026, wherein it is noted Sanchez does not teach “sequentially rendering”. Specifically, the examiner still maintains that “sequentially rendering” is likely not taught in the Sanchez, specifically given the narrower interpretation that “sequentially rendering” means rendering only a first region (i.e., nothing of a second and third region), followed by a second region (i.e., only after the first region is fully rendered) and so forth. Given the broader interpretation to include “sequentially rendering” as, for example, rendering the first, second, and third region in tandem, but completing the first, second, and third regions in order during rendering, the examiner notes Sanchez would teach “sequentially rendering” ([pg. 4, Fig. 4, see specifically far right column with Face ROI and progressive rendering] specifically note that the mans face is fully rendered, before fully rendering the women face). For the sake of compact prosecution, the examiner has maintained “sequentially rendering” as analogous to the first narrower interpretation, and thus Sanchez fails to specifically disclose “sequentially rendering”, though it discloses the remainder of the amended language. Therefore, the rejections have been withdrawn. However, upon further consideration, a new ground of rejection is made in view of “Saliency enabled compression in JPEG framework” to Rahul et al., and further in view of U.S. Publication No. 2018/0288423 to Vembar et al, which teaches “sequentially rendering” analogous to the first interpretation ([par. 0175, ln. 1-23], [par. 0176, ln. 1-8] see 103 for full citations). The examiner further notes that Vembar likewise teaches sequentially ordered regions including a first region which is associated with the bounding region and is a most salient portion of the encoded image, a plurality of second regions which concentrically surround the first region, and a plurality of third regions which concentrically surround the plurality of second regions, and progressively rendering a decoded version of the encoded image by sequentially rendering the first region, the plurality of second regions, and the plurality of third regions, wherein the first region is rendered with a higher resolution than the plurality of second regions, and the plurality of third regions. ([Fig. 6E-L], [par. 0151, ln. 1-21], [par. 0152, ln. 1-9], [par. 0153, ln. 1-14], [par. 0175, ln. 1-23], [par. 0176, ln. 1-8] see 103 for full citations).
Claim Rejections - 35 USC § 101
6. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
7. Claim 29 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim does not fall within at least one of the four categories of patent eligible subject matter because “…computer readable medium…” is not defined in such a manner to preclude transitory computer readable medium (e.g., signals or waveforms, see MPEP 2016.03(I)). Specifically, while the specifications define computer readable medium to include non-transitory medium, they do not define computer readable medium to exclude transitory medium. The examiner recommends amending “non-transitory” before “computer readable medium”, such that the claim recites “…non-transitory computer readable medium…”. This would preclude transitory computer readable medium from the BRI of the claim and obviate the 101 rejection.
Claim Rejections - 35 USC § 103
8. The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
9. Claims 1-2, 4-6, 13-14, 24, 28, and 29 are rejected under 35 U.S.C. 103 as being unpatentable over “Saliency enabled compression in JPEG framework” to Rahul et al. (hereinafter Rahul) and further in view of in view of U.S. Publication No. 2018/0288423 to Vembar et al. (hereinafter Vembar).
10. Regarding Claim 1, Rahul discloses a computer-implemented method, comprising ([pg. 1142, Abstract, par. 1, ln. 1-4] “Through this paper, a novel region-of-interest (ROI) dependent quantization method in JPEG framework is proposed. The proposed method judiciously quantizes DCT coefficients belonging to salient and non-salient regions of the image. In this work, multiple ROIs are optimally identified and ranked by using variances”):
receiving, via a computing device ([pg. 1142, col. 1, 1 Introduction, par. 1, ln. 1-7] “Usage of image data through the Internet has exponentially increased among the users [1]. Compression is essentially required to manage this high data rate of images without degrading the quality to an unacceptable level. The necessity of accessing the high definition images with quality as of paramount importance has become the major issue in designing such algorithms to operate in real time.”), a plurality of bytes of an encoded image, wherein the encoded image comprises a salient portion ([pg. 1143, Fig. 1, Fig. 2], [pg. 1143, col. 1, par. 2, ln. 1-29] “The work done in multi-level saliency-based compression techniques [15, 21, 26, 27] exhibits an improved trade-off between CR and perceptual quality than using only two-level saliency… JPEG 2000 standard [26] incorporates both two-level and multi-level models of ROI encoding using maximum shift or MAXSHIFT and general scaling based method, respectively. The major challenge in multi-level saliency-based compression technique is the requirement of sending the overhead for the shape of salient regions and their ranks used to grade the saliency. The overhead is proportional to the complexity of the shape and the number of ranks. This is because the complex shaped regions will require a larger number of model parameters which will increase the overhead. Also, the overhead for ranks information will increase with an increase in the number of salient regions, i.e. for the R number of ranks, ⌈log2R⌉ bits per rank will be needed. If the ROI mask is generated for an arbitrary shaped ROI, the decoder needs to reproduce the ROI mask [26], making the decoder computationally complex and increased memory requirement on the decoder side. To reduce the requirement of the overhead information and to make the decoder simple, the ROI shape is approximated as a rectangular box [15, 26], as shown in Fig. 1. The coordinates of the opposite vertices of the rectangular boxes and their rank information are sent to the decoder. This approach saves the overhead information to a good extent but the CR is compromised, as the actual ROI has been approximated by a rectangular bounding box”, [pg. 1144, 3 Proposed method, par. 1, ln. 1 to pg. 1145 to col. 2, 3.1 Number of regions identification and multiple saliency identification, par. 5, ln. 5] “The encoder processes a given image through two paths. The first path generates multi-level saliency map for the input image. The optimal number of classes is adaptively calculated by using efficiency of segmentation [28]. The image is then segmented into the same number of salient classes by maximising between-class variances [29]. Every class is then given a rank based on its importance by using their weighted variances. The class having high weighted variance is given higher rank, i.e. more importance, and vice versa. The second path is used for the adaptive quantisation. The image is decomposed into blocks of size 8 × 8 and each block is ranked based on the input saliency map obtained from the first path by applying probability bound. 2D-DCT coefficients of each block are quantised adaptively by the quantisation parameters modified by the rank of the block. The higher rank block (i.e. more salient) will be lightly quantised and vice versa. The quantised coefficients are then entropy coded and the overhead for the rank of the blocks is reduced by using delta encoding method [30]… Unlike method in [26], where decoder requires reproducing the ROI mask, making the decoder complex, the decoder of our method is simple as ROI information is sent to the decoder by the encoder. The reconstruction of the image is the inverse of the encoding steps. The detailed description of the key steps in the encoding process is given as follows… Salient regions are identified by segmenting the image into Knumber of classes, to be discussed later, by maximising between-class variances. For segmenting the image, the Otsu's segmentation method [28, 29, 31] is extended for K classes. Let these K classes be arbitrary bounded by
K
+
1
intensity levels
(
t
0
,
t
1
,
t
2
,
…
,
t
K
)
as
t
0
<
t
1
<
t
2
<
…
<
t
K
-
1
<
t
k
. For an image with
L
intensity levels,
t
i
(
0
≤
i
≤
K
)
is intensity value of pixels with
t
0
=
0
, and
t
K
=
L
-
1
. Let
i
th class
(
1
<
i
<
K
-
1
)
consist of all the pixels with intensities in the range
[
t
i
-
1
,
t
i
-
1
]
. Whereas, the Kth class consists of pixels with intensity values in the range of
[
t
K
-
1
,
t
K
]
. With theses initial assumptions, probability of the ith class occurrence
(
ω
i
)
, and the class mean
(
μ
i
)
are obtained das follows
ω
i
=
∑
j
=
t
i
-
1
t
i
p
j
,
μ
i
=
1
ω
i
∑
j
=
t
i
-
1
t
i
j
p
j
,
μ
T
=
∑
i
=
1
K
ω
i
μ
i
(4). Here
p
j
is the probability of the pixels with intensity value j, and
(
μ
T
)
is the mean of the image. Thereafter between-class variance
(
σ
K
2
)
can be obtained using the following equation:
σ
K
2
=
∑
i
=
1
K
ω
i
(
μ
i
-
μ
T
)
2
(5)
σ
K
2
is the function of
ω
i
and
μ
i
and these parameters, in turn, are functions of the chosen class boundaries
t
1
,
t
2
,
…
,
t
K
-
1
. It is desired to obtain an optimal set of the class boundaries that result into a maximum value of
σ
K
2
. This can be obtained by iteratively solving (5) for possible Boundary values given in (4). The maximum value of
σ
K
2
is called maximum between-class variance. To identify the total number of classes (K), it is proposed to first obtain the goodness-of-segmentation (GOS)
(
η
K
)
, given in (6). Between-class variance,
σ
K
2
, given in (5), and weighted variance,
S
i
, given in (8), are used to calculate the total variance
σ
T
2
and
η
K
, for initial value of K=2. Value of K is incremented till the inequality given in (7) is satisfied for required value of
(
η
r
)
, typically in the range of 0.8-0.99
σ
T
2
=
σ
K
2
+
∑
i
=
1
K
S
i
,
η
K
=
σ
K
2
σ
T
2
(6) As it can be observed from (6),
η
K
will be <1
η
K
≥
η
r
(
0
≤
η
r
≤
1
)
(7) Choosing number of classes (K) based on GOS helps to avoid the over-segmentation and under-segmentation situations…”);
determining a bounding region for the encoded image, wherein the bounding region is indicative of a location of the salient portion in the encoded image ([pg. 1143, Fig. 1, Fig. 2], [pg. 1143, col. 1, par. 2, ln. 1-29], [pg. 1145, Fig. 5], [pg. 1145, col. 1, 3.3 Block ranking, par. 2, ln. 1 to col. 2, par. 2, ln. 3] “After classifying pixels based on the threshold values
t
1
,
t
2
,
…
,
t
K
-
1
and ranking them according to sorted series of
S
i
, given in (8), we propose to use probability mass function to rank every
8
×
8
blocks used in JPEG. Blocks having pixels with more than one rank may be the one that is on the border, or sometimes on the edges. The block is assigned rank r, whenever the empirically proposed probability bound (10) is satisfied, starting with r=1
∑
i
=
1
r
p
i
≥
1
K
-
r
+
1
,
r
=
1,2
,
…
,
K
(10) where
p
i
is the probability of the ith ranked pixels in the block. To illustrate the probability bound and ranking of blocks, let us assume K=4 and apply (10) on four different blocks of size
3
×
3
shown in Figs. 6a-d. Considering Fig. 6a, for example, it is found that
p
1
=
0.33
,
p
2
=
p
3
=
0
, and
p
4
=
0.66
. The probability bound (10) is then applied, starting with
r
=
1
. The probability bound is satisfied for
r
=
1
, resulting block rank to be 1. Similarly, blocks in Fig. 6b-d get ranks 2, 3, and 4 respectively… DCT coefficients of every ranked block of size
8
×
8
are adaptively quantised as per their importance, estimated in terms of their rank values (r). The quantisation table
(
T
50
)
used in JPEG baseline [10] is proposed to be scaled by a factor
F
r
for rth ranked block
(
1
≤
r
≤
K
)
, and the same is controlled by two variables
V
a
r
and
Q
a
m
given as follows:
F
r
=
V
a
r
+
(
r
-
1
)
Q
a
m
(11)
F
r
=
V
a
r
for
r
=
1
, i.e. for the most salient blocks and value of
F
r
increases by
(
r
-
1
)
Q
a
m
as the saliency of the block decreases (i.e. r increases)…”, [pg. 1147, Fig. 10], [pg. 1147, col. 1, par. 1, ln. 1-10] “Fig. 10 shows the comparison in terms of accuracy of reconstructed ROI at the decoder side, between the proposed method of sending the ROI and the rectangular approximation used in state-of-art saliency enabled methods [15, 26]. The reference images can be seen in Fig. 1. It is observed that the proposed approach of sending ROI information to the decoder by using block ranks retains the ROI structure better than the rectangular approximation of ROI. The average overhead found to be0.00038 bpp while using the rectangular approximation, and 0.0091 bpp while using the proposed method”); and
partitioning the encoded image into a plurality of sequentially ordered regions including a first region which is associated with the bounding region and is a most salient portion of the encoded image, a plurality of second regions {which concentrically surround the first region}, and a plurality of third regions {which concentrically surround the plurality of second regions} ([pg. 1145, Fig. 5], [pg. 1144, 3 Proposed method, par. 1, ln. 1 to pg. 1145 to col. 2, 3.1 Number of regions identification and multiple saliency identification, par. 5, ln. 5], [pg. 1145, col. 1, 3.3 Block ranking, par. 2, ln. 1 to col. 2, par. 2, ln. 3], [pg. 1146, col. 2, par. 1, ln. 2-16] “When any DWT or DCT methods in [6, 10–12, 14] are applied on the most important regions (r = 1), the mean-square error (MSE) of these regions is higher than overall MSE. The reason for this is while applying these transform-based methods in a region with high variance, the energy compaction is lesser compared to a region with lower variance [18], which results in higher MSE after quantisation to achieve lower bit-rate. A similar example can be referred from rate-distortion curve in Fig. 7, where the PSNR after applying JPEG on the most important regions of the image is always lower than the overall image. This information suggests that the PSNR values provided in Table 3 for the methods in [6, 11, 12, 14], which is for the whole image will have a lower value of PSNR for the regions with (r = 1). It is clear that the proposed method outperforms those reported in [6, 10–12,14]”, [pg. 1147, Table 2, see
R
1
final column], [pg. 1144, col. 2, 3.2 Saliency ranking, par. 1, ln. 1 to pg. 1145, col. 1, par. 1, ln. 9] “In order to rank each class, consisting of randomly distributed pixels, weighted variance of the pixels are obtained, using the following equation:
S
i
=
ω
i
σ
i
2
,
i
=
1,2
,
…
,
K
(8) where
σ
i
2
is the ith class, as given in the following equation:
σ
i
2
=
1
ω
i
∑
j
=
t
i
-
1
t
i
(
j
-
μ
i
)
2
p
j
(9)
S
i
for
1
≤
i
≤
K
is sorted in descending order. The ith class pixels will get rank q where q is the position of the sorted
S
i
. Highest weighted variance referrers to the most salient class and gets the highest rank and vice versa, i.e. pixels corresponding to max(
S
i
) will get (r=1) and min(
S
i
) will be ranked (r=K). The main aim is to give more importance to the class with considerable area having high variance. As expressed in (8), a class with a high variance but very small area (small
ω
i
) may get less importance than a class with a relatively lower variance but larger area.”);
and {progressively} rendering a decoded version of the encoded image {by sequentially rendering the first region, the plurality of second regions, and the plurality of third regions}, wherein the first region is rendered with a higher resolution than the plurality of second regions and the plurality of third regions ([pg. 1145, Fig. 5], [pg. 1144, 3 Proposed method, par. 1, ln. 1 to pg. 1145 to col. 2, 3.1 Number of regions identification and multiple saliency identification, par. 5, ln. 5], [pg. 1145, col. 1, 3.3 Block ranking, par. 2, ln. 1 to col. 2, par. 2, ln. 3], [pg. 1146, col. 2, par. 1, ln. 2-16], [pg. 1147, Table 2, see
R
1
final column]).
One of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Rahul specifically ranks the saliency regions and quantizes them to a lesser extent the lower the rank they have (e.g., region ranked r=1 is most salient and quantized/encoded to a lesser extent then region ranked r=2), and therefore, Rahul discloses rendering a high-resolution version of the first region, and a lower resolution for the second and third regions, and so forth. With regard to “receiving, via a computing device” one of ordinary skill in the art, before the effective filling date of the claimed invention, would specifically recognize that the encoding and decoding method of Rahul is intended for transmission of images across the internet and/or between computing devices (e.g., server to client, downloading an image online, etc.), and as such, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, that the image would be received, via a computing device. However, Rahul does not specifically disclose progressively rendering an image by sequentially rendering the first region, the plurality of second regions, and the plurality of third regions, or wherein the second region concentrically surrounds the first region, and the third region concentrically surround the plurality of second regions.
However, Vembar teaches progressively rendering a decoded version of the encoded image by sequentially rendering the first region, the plurality of second regions, and the plurality of third regions, wherein the first region is rendered with a higher resolution than the plurality of second regions and the plurality of third regions ([par. 0153, ln. 1-14] “In another example of reducing precision, a formula may be applied based on the location of the target pixel relative to the focus region. For example, the system may calculate the shortest distance from the target pixel to the focus region boundary and reduce the precision proportionally to the calculated distance. Alternatively, a particular pixel may be selected as the focal point (e.g. a focal pixel) and the distance may be calculated from the target pixel to the focal pixel. The system may use a linear formula, a non-linear formula (e.g. parabolic), or other suitable formula for the proportional precision reduction. The system may also maintain a set of ranges for the precision reduction (e.g. 0 to 100 pixels [no reduction]; 101 to 300 pixels [20% reduction]; 301 pixels or more [50% reduction]).”, [par. 0175, ln. 1-23] “Turning now to FIG. 7H, some embodiments may advantageously provide partial frame buffer transfers and prioritized rendering. For a frame 778 having five viewports prioritized 1 through 5, the system may render the viewports in the priority order. The system may start sending the first viewport as soon as it is done rendering (e.g. without waiting for the next viewport to render). The same render/send process continues for each of the viewports. The full set of viewports may be merged in the display. For example, encoding may be performed utilizing High Efficiency Video Coding (HEVC) tiles or multi-view HEVC. For subsequent frames, the system may progressively re-prioritize and/or re-size the viewports starting from an identified area of importance. For example, the system may use eye tracker information to identify a portion of the frame to prioritize. The system may also change the priorities based on motion (e.g. if the user is turning their head or body), content (e.g. where the application wants the user to focus such as something more interesting happening that the application wants the user's attention on), and/or motion prediction (e.g. if the user is turning their head, the system may prioritize a side region which will be coming into view based on the prediction that the user will continue to turn their head in the same direction).”, [par. 0176, ln. 1-8] “For example, the system may render the first priority viewport first, at high resolution. The system may then render the second priority viewport next, at medium resolution. The system may then render the remaining prioritized viewports in order, at low resolution. Advantageously, the system may start transmitting the first rendered section while rendering the second section. The receiving unit (e.g. HMD) can start decoding the first section as soon as it is received.”), and further teaches wherein the second region concentrically surrounds the first region, and wherein the third region concentrically surrounds the plurality of second regions ([Fig. 6E-L], [par. 0151, ln. 1-21] “Turning now to FIGS. 6E to 6J, embodiments of color masks may be represented by any of a variety of two or more successively surrounded, non-intersected regions. While the regions illustrated together, a mask for each may be defined and applied separately. The masks may have any shape such as circular (e.g. FIGS. 6E and 6G), elliptical (e.g. FIGS. 6F, 6G, and 6H), square or rectangular (e.g. FIG. 6I), or arbitrary (e.g. FIG. 6J). The inner most region is generally uncompressed (e.g. 0% compression), but may have compression applied in some use cases (e.g. a power saving setting, see FIG. 6J with 10% compression). The inner most region may be surrounded by one or more successive, non-intersecting regions with successively more compression applied for each successive region (e.g. further away from the focus area). The regions may have a common center (e.g. FIGS. 6E and 6F) or no common center (e.g. FIGS. 6H and 6J). The orientation of the masks may be aligned with the display (e.g. FIGS. 6E, 6F, 6G, and 6I) or not (e.g. FIGS. 6H and 6J). The shape of each region may be the same (e.g. FIGS. 6E, 6F, and 6H) or may be different from each other (e.g. FIGS. 6G, 6I, and 6J).”, [par. 0152, ln. 1-9] “Turning now to FIGS. 6K and 6L, an embodiment of a color mask 650 may be applied to an image area 652 based on a focus area. For example, if the focus area is roughly centered (e.g. if static if that is where the user is looking), the color mask may also be roughly centered when applied to the image area (e.g. see FIG. 6K). If the focus area moves (e.g. based on gaze information, motion information, content, etc.), the color mask may likewise move based on the new focus area (e.g. see FIG. 6L).”, [par. 0153, ln. 1-14]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would specifically recognize that the focus regions as taught in Vembar are directly analogous to a salient region. Furthermore, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Vembar and Rahul as within the same field of encoding and decoding images based on saliency, and as analogous to the claimed invention. The motivation to combine is disclosed in Vembar, wherein it provides an improvement for network speed and can improve user experience ([par. 0149, ln. 1-16] “…Wireless applications may particularly benefit from compression of the color representation. For example, the screen may be compressed away from the center (or area of focus). Using less bits in the color representation may improve network speed. By using full precision may in the focus region, some embodiments efficiently dedicate encoding resources on a region of the screen that matters most to the user.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Rahul with the progressive image rendering and concentric regions of Vembar through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the method of Rahul with the progressive image rendering and concentric regions of Vembar to obtain the invention as specified in claim 1.
11. Regarding Claim 2, a combination of Rahul and Vembar teaches the method of claim 1. Rahul further discloses wherein the bounding region is square shaped ([pg. 1143, Fig. 1], [pg. 1147, Fig. 10], [pg. 1145, col. 1, 3.3 Block ranking, par. 2, ln. 1 to col. 2, par. 2, ln. 3]). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that that Rahul discloses dividing the image into bounding regions comprising a multitude of 8x8 squares, or likewise, that a square shape can be used to denote the bounding region with reduced overhead ([pg. 1147, col. 1, par. 1, ln. 1-10]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the method of Rahul with the progressive image rendering and concentric regions of Vembar to obtain the invention as specified in claim 2.
12. Regarding Claim 4, a combination of Rahul and Vembar teaches the method of claim 1. Rahul further discloses wherein the salient portion is identified during encoding of the image ([pg. 1143, Fig. 1, Fig. 2], [pg. 1143, col. 1, par. 2, ln. 1-29] [pg. 1144, 3 Proposed method, par. 1, ln. 1 to pg. 1145 to col. 2, 3.1 Number of regions identification and multiple saliency identification, par. 5, ln. 5]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the method of Rahul with the progressive image rendering and concentric regions of Vembar to obtain the invention as specified in claim 4.
13. Regarding Claim 6, a combination of Rahul and Vembar teaches the method of claim 1. Rahul further discloses wherein the salient portion is identified at the time the image is received ([pg. 1145, Fig. 5, see Original Image and Path 1 performed prior to encoding]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would specifically recognize that to identify salient portions of the image as disclosed in Rahul are determined prior to encoding but after having received an image, and therefore, it would have been obvious to one of ordinary skill in the art to identify the salient portion at the time the image is received. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the method of Rahul with the progressive image rendering and concentric regions of Vembar to obtain the invention as specified in claim 6.
14. Regarding Claim 13, a combination of Rahul and Vembar teaches the method of claim 1. Rahul further discloses wherein the salient portion is a first salient portion, wherein the image comprises a second salient portion, and wherein the determining of the bounding region comprises determining the bounding region so as to prioritize rendering of the first salient portion and the second salient portion ([pg. 1147, Fig. 9 and 10], [pg. 1144, 3 Proposed method, par. 1, ln. 1 to pg. 1145 to col. 2, 3.1 Number of regions identification and multiple saliency identification, par. 5, ln. 5], [pg. 1144, col. 2, 3.2 Saliency ranking, par. 1, ln. 1 to pg. 1145, col. 1, par. 1, ln. 9], [pg. 1145, col. 1, 3.3 Block ranking, par. 2, ln. 1 to col. 2, par. 2, ln. 3]). Specifically, one of ordinary skill in the art, before the effective filing date of the claimed invention, would recognize Rahul contains multiple salient portions where the rendering of a first salient portion is given priority when its rank is lower, because it is quantizing to a lesser extent. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the method of Rahul with the progressive image rendering and concentric regions of Vembar to obtain the invention as specified in claim 13.
15. Regarding Claim 14, a combination of Rahul and Vembar teaches the method of claim 1. Rahul further discloses wherein the determining of the bounding region comprises selecting, form a plurality of candidate bounding regions, a bounding region that prioritizes rendering of the salient portion ([pg. 1144, 3 Proposed method, par. 1, ln. 1 to pg. 1145 to col. 2, 3.1 Number of regions identification and multiple saliency identification, par. 5, ln. 5], [pg. 1144, col. 2, 3.2 Saliency ranking, par. 1, ln. 1 to pg. 1145, col. 1, par. 1, ln. 9]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the method of Rahul with the progressive image rendering and concentric regions of Vembar to obtain the invention as specified in claim 14.
16. Regarding Claim 24, a combination of Rahul and Vembar teaches the method of claim 1. Rahul discloses wherein the {progressively} rendering of the decoded version of the encoded image comprises pixel-wise rendering ([pg. 1144, 3 Proposed method, par. 1, ln. 1 to pg. 1145 to col. 2, 3.1 Number of regions identification and multiple saliency identification, par. 5, ln. 5], [pg. 1145, col. 1, 3.3 Block ranking, par. 2, ln. 1 to col. 2, par. 2, ln. 3] see 8x8 bocks), and the method further comprises rendering a first number of pixels in a portion inside the bounding region, and a second number of pixels in the portion outside of the bounding region, wherein the first number is {greater than} the second number ([pg. 1145, Fig. 10], [pg. 1144, 3 Proposed method, par. 1, ln. 1 to pg. 1145 to col. 2, 3.1 Number of regions identification and multiple saliency identification, par. 5, ln. 5], [pg. 1145, col. 1, 3.3 Block ranking, par. 2, ln. 1 to col. 2, par. 2, ln. 3]). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that Rahul discloses a pixel-wise rendering in 8x8 blocks, and a first and second region comprising a first number and a second number of pixels, but does not specifically disclose that the first number is greater then the second number (i.e., the first region has more pixels then the second region). Likewise, Rahul does not specifically disclose progressively rendering the image.
However, Vembar teaches to progressively render the image, and wherein the first region contains a first number of pixels greater than the second number of pixels contained in the second region ([par. 0153, ln. 1-14], [par. 0175, ln. 1-23], [par. 0176, ln. 1-8]). The motivation to combine is analogous to claim 1. One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Rahul with the progressive image rendering and higher resolution concentric regions of Vembar through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Rahul with the progressive image rendering and higher resolution concentric regions of Vembar to obtain the invention as specified in claim 24.
17. Regarding Claim 28, the claim language is analogous to claim 1, with the exception of “A computing device, comprising: one or more processors; and data storage, wherein the data storage has stored thereon computer-executable instructions, that when executed by one or more processors, cause the computing device to carry out functions comprising:” wherein the remainder of the claim is analogous to claim 1. Rahul does not specifically disclose one or more processors, or a data storage unit, though it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, that Rahul discloses a method intended to be performed on a computer ([pg. 1142, Abstract, par. 1, ln. 1-4], [pg. 1142, col. 1, 1 Introduction, par. 1, ln. 1-7]).
However, Vembar teaches a computing device, comprising: one or more processors; and data storage, wherein the data storage has stored thereon computer-executable instructions, that when executed by one or more processors, cause the computing device to carry out functions of the method ([par. 0141, ln. 1-19] “… processor 611, persistent storage media 612, PPU 613, image compressor apparatus 614/620, focus identifier 621, color compressor 622, frame buffer 623, mask store 624, and other components may be implemented in hardware, software, or any suitable combination thereof. For example, hardware implementations may include configurable logic such as, for example, programmable logic arrays (PLAs), FPGAs, complex programmable logic devices (CPLDs), or in fixed-functionality logic hardware using circuit technology such as, for example, ASIC, complementary metal oxide semiconductor (CMOS) or transistor-transistor logic (TTL) technology, or any combination thereof. Alternatively, or additionally, these components may be implemented in one or more modules as a set of logic instructions stored in a machine- or computer-readable storage medium such as random access memory (RAM), read only memory (ROM), programmable ROM (PROM), firmware, flash memory, etc., to be executed by a processor or computing device.”). The motivation to combine would have been obvious to one of ordinary skill in the art, in that the method of Rahul is intended to be implemented on a computer, and the device of Vembar would allow for the implementation of the method. One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Rahul with the progressive image rendering, concentric regions, and computing device of Vembar through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Rahul is intended to be implemented on a computer, and the device of Vembar to obtain the invention as specified in claim 28.
18. Regarding Claim 29, the claim language is analogous to claim 1, with the exception of “An article of manufacture comprising one or more computer readable media having computer-readable instructions stored thereon that, when executed by one or more processors of a computing device, cause the computing device to carry out functions that comprise:”, wherein the remainder of the claim is analogous to claim 1. Rejections analogous to claim 28 are further applicable to claim 29. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Rahul with the progressive image rendering, concentric regions, and computing device of Vembar to obtain the invention as specified in claim 29.
19. Claim 3 and 30 are rejected under 35 U.S.C. 103 as being unpatentable over “Saliency enabled compression in JPEG framework” to Rahul, in view of U.S. Publication No. 2018/0288423 to Vembar, and further in view of “JPEG XL next-generation image compression architecture and coding tools” to Alakuijala et al. (hereinafter Alakuijala).
20. Regarding Claim 3, a combination of Rahul and Vembar teaches the method of claim 1. Rahul teaches wherein the encoded image is encoded in a JPEG{-XL} format ([pg. 1142, Abstract, par. 1, ln. 1-4]). Rahul and Vembar do not specifically disclose wherein the format is JPEG-XL.
However, Alakuijala teaches wherein the encoding can be in JPEG-XL format ([pg. 1, Abstract, par. 1, ln. 1-8] “An update on the JPEG XL standardization effort: JPEG XL is a practical approach focused on scalable web distribution and efficient compression of high-quality images. It will provide various benefits compared to existing image formats: significantly smaller size at equivalent subjective quality; fast, parallelizable decoding and encoding configurations; features such as progressive, lossless, animation, and reversible transcoding of existing JPEG; support for high-quality applications including wide gamut, higher resolution/bit depth/dynamic range, and visually lossless coding. Additionally, a royalty-free baseline is an important goal. The JPEG XL architecture is traditional block-transform coding with upgrades to each component…”). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that Rahul, Vembar, and Alakuijala are within the same field of encoding, decoding, and rendering of images, and Rahul and Alakuijala as within the same field of JPEG images, and as analogous to the claimed invention. The motivation to combine is disclosed in Alakuijala, wherein JPEG-XL provides wide gamut, higher resolution/bit depth/dynamic range, and visually lossless coding as compared to standard JPEG ([pg. 1, Abstract, par. 1, ln. 1-8]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Rahul with the progressive image rendering and concentric regions of Vembar, and further combined the method of the combination of Rahul and Vembar with the JPEG-XL format of Alakuijala, through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the method of Rahul with the progressive image rendering and concentric regions of Vembar and the JPEG-XL format of Alakuijala to obtain the invention as specified in claim 3.
21. Regarding Claim 30, a combination of Rahul and Vembar teaches the method of claim 1. Rahul does not specifically teach wherein the encoded image is a JPEG-XL format, and sequentially rendering the first region, the plurality of second regions, and the plurality of third regions comprises applying a first smoothing process to a first boundary between the first region and the plurality of second regions and applying a second smoothing process to a second boundary between the plurality of second regions and the plurality of third regions.
However, Vembar teaches sequentially rendering the first region, the plurality of second regions, and the plurality of third regions ([Fig. 6E-L], [par. 0151, ln. 1-21], [par. 0152, ln. 1-9], [par. 0153, ln. 1-14], [par. 0175, ln. 1-23], [par. 0176, ln. 1-8]). The motivation to combine remains analogous to claim 1. One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Rahul with the progressive image rendering and concentric regions of Vembar through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Vembar does not specifically disclose wherein the encoded image is a JPEG-XL format, or applying a first smoothing process to a first boundary between the first region and the plurality of second regions and applying a second smoothing process to a second boundary between the plurality of second regions and the plurality of third regions. Therefore, a combination of Rahul and Vembar does not specifically disclose wherein the wherein the encoded image is a JPEG-XL format, or applying a first smoothing process, or a second smoothing process.
However, Alakuijala specifically teaches wherein the encoded image is a JPEG-XL format, and rendering comprises applying a {first} smoothing process {to a first boundary between the first region and the plurality of second regions} and applying a {second} smoothing process {to a second boundary between the plurality of second regions and the plurality of third regions} ([pg. 1, Abstract, par. 1, ln. 1-8], [pg. 11, 4.9 Loop filters, par. 1, ln. 1 to par. 3, ln. 7] “Despite the significant improvements that JPEG XL delivers to reduce artefacts, block boundaries and ringing can still be noticed, especially at somewhat lower qualities. Ringing artefacts are spurious signals near sharp edges, caused by quantizing or truncating high-frequency components (see Figure 7). To mitigate their impact, JPEG XL employs two different loop filters that are applied to the image after the decompression process. The first loop filter is a smoothing convolution. As smoothing the image inherently introduces a sharpness loss, this effect is compensated by the encoder by means of a sharpening filter that is applied before the DCT step. The overall effect of this procedure is that the visual impact of block boundaries gets reduced, while still preserving sharp details present in the original image. The second loop filter is intended to reduce ringing, while still preserving texture that is transmitted in the image. To achieve this effect, it applies an adaptive smoothing algorithm related to Non-Local Means,16 with some modifications to improve processing speed. To further improve the detail preservation of this filter, the JPEG XL format can require the decoder to apply a quantization constraint to the output of the filter: the DCT step is applied again, and the decoder ensures that the resulting coefficients are inside the range of values that would have been quantized to the coefficient read from the bitstream by clamping. The decoder then applies an IDCT step again to produce the final output of the filter”). The motivation to combine remains analogous to claim 3, and is further discloses in Alakuijala, wherein the smoothing process prevents artifacts ([pg. 11, 4.9 Loop filters, par. 1, ln. 1 to par. 3, ln. 7]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Rahul with the progressive image rendering and concentric regions of Vembar, and further combined the method of the combination of Rahul and Vembar with the JPEG-XL and smoothing of Alakuijala through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
However, Alakuijala fails to specifically disclose wherein the smoothing process comprises a first smoothing process and a second smoothing process, or wherein they are applied specifically to the boundaries of the regions. Therefore, a combination of Rahul, Vembar, and Alakuijala fails to specifically discloses a first smoothing process and a second smoothing process applied to a first and second boundary.
However, Agrawal specifically teaches wherein a first smoothing process is applied to a first boundary between the first region and the plurality of second region, and a second smoothing process is applied to a second boundary between the plurality of second regions and the plurality of third regions ([Fig. 1A, see layer 103, 105, 106], [col. 28, ln. 1 to col. 29, ln. 5] “…In order to reduce the visually unappealing “cut-and-paste” effect of simply blurring background image data while keeping foreground image data in focus, using the simplified approach described above, a multi-layer approach for depth of field effect, where blur is gradually increased away from the segmentation boundary, is described below. The simplified approach described above can be conceptually considered as a “two layer approach”, with a foreground layer and a background layer. If the final segmentation image does not include errors, the transition between foreground and background occurs precisely on the segmentation boundary 107, giving a visually pleasing rendering. However, due to segmentation errors (e.g., erosion and/or dilation), the foreground-background transition may not occur on the actual user boundary and visual artifacts may result. Using a multi-layer approach, additional layers are included to smooth out the transition from sharply focused foreground image data to blurry background image data. According to an example technique for multi-layer blending of foreground and background image data, N layers may be defined, with each layer comprising an associated transition width (w.sub.i) and a blur value b.sub.i, as depicted in FIG. 1A. As previously described, a blur value b.sub.i may represent the σ value of a low pass filter (e.g., a Gaussian filter) used to generate a weighted average of a contiguous neighborhood of pixels and apply that weighted average as the new value for a pixel being blurred using the low pass filter. In general, the larger the blur value b.sub.i, the larger the neighborhood of pixels and more the pixels will be blurred using the particular filter. In general, a low pass filter used to blur pixels may be effective to reduce the high frequency components of pixels by generating a weighted average of the contiguous neighborhood of pixels, as specified by the σ value of the low pass filter… Accordingly, each of the layers (e.g., layers 103, 105, 106 in FIG. 1A) is blended together with the previous layer and the blend strength of the layer is adjusted within each layer to prevent sharp transitions in the amount of blur between any two layers. Blend strength F.sub.k is 1 at the beginning of a layer and is 0 at the start of the next layer when traversing the layers in FIG. 1A from the segmentation boundary 107 outwards. A disc blur and/or Gaussian filter may be used to blur the pixels values according to the blur value b.sub.i. Various other multi-layer blending algorithms may be used in accordance with the present disclosure. For example, non-linear methods may be used to compute blending strength. Additionally, different equations may be used to blend various blurred layers together. Furthermore, any number of layers may be used in accordance with the multi-layer blending techniques described herein. In addition to the multi-layer blending described above, the boundary of the foreground image data (e.g., the image data encoded with foreground indicator data) may be blurred with a small-sized (e.g., a σ value of 1, 2, 3, etc.) median filter before composing the foreground image data with the multi-layer blended background image data. A small-sized median filter may act as a directional filter that may maintain boundary edges at the segmentation boundary 107 while suppressing noise, leading to improved blending between foreground image data and background image data and reducing visible artifacts due to image segmentation errors.”). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Rahul, Vedmar, and Agrawal as within the same field of image processing for segmentation of focus areas and encoding, and as analogous to the claimed invention. The motivation to combine is disclosed in Agrawal, wherein it reduces the effects of noise between the adjacent regions ([col. 28, ln. 1 to col. 29, ln. 5]). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that the concentric regions of Agrawal are directly analogous to the concentric regions of the combination of Rahul, Vedmar, and Alakuijala, and it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to have incorporated an analogous smoothing mechanism to the borders of the regions as taught in Agrawal such that a first smoothing is applied to a boundary between regions one and two and a second smoothing is applied to a second boundary between regions two and three. One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Rahul with the progressive image rendering and concentric regions of Vembar and the JPEG-XL and smoothing of Alakuijala, and further combined the method of the combination of Rahul, Vembar, and Alakuijala with the first and second smoothing to the first and second boundaries as taught in Agrawal, through known means, with no change to their respective function, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Rahul with the progressive image rendering and concentric regions of Vembar, the JPEG-XL and smoothing of Alakuijala, and the first and second smoothing to the first and second boundaries as taught in Agrawal to obtain the invention as specified in claim 30.
22. Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over “Saliency enabled compression in JPEG framework” to Rahul, in view of U.S. Publication No. 2018/0288423 to Vembar, and further in view of “Learning to Detect A Salient Object” to Liu et al. (hereinafter Liu).
23. Regarding Claim 7, a combination of Rahul and Vembar teaches the method of claim 6. Rahul and Vembar do not specifically disclose wherein the salient portion is identified by applying a trained machine learning model.
However, Liu teaches wherein the salient portion is identified by applying a trained machine learning model ([pg. 1, col. 1, Abstract, par. 1, ln. 1-9] “We study visual attention by detecting a salient object in an input image. We formulate salient object detection as an image segmentation problem, where we separate the salient object from the image background. We propose a set of novel features including multi-scale contrast, center-surround histogram, and color spatial distribution to describe a salient object locally, regionally, and globally. A Conditional Random Field is learned to effectively combine these features for salient object detection.”, [pg. 3, col. 1, CRF for Salient Object Detection, par. 1, ln. 1 to col. 2, par. 2, ln. 7] “We formulate the salient object detection problem as a binary labeling problem by separating the salient object from the background. In the Conditional Random Field (CRF) framework [13], the probability of the label
A
=
{
a
x
}
given the observation image I is directly modeled as a conditional distribution
P
A
I
=
1
z
e
x
p
(
-
E
(
A
|
I
)
)
, where Z is the partition function. To detect a salient object, we define the energy
E
(
A
|
I
)
as a linear combination of a number of K salient features
F
k
(
a
x
,
I
)
and a pairwise feature
S
a
x
,
a
x
'
,
I
:
E
A
I
=
∑
x
∑
k
=
1
K
λ
k
F
k
a
x
,
I
+
∑
x
,
x
'
S
a
x
,
a
x
'
,
I
, (3) where
λ
k
is the weight of the kth feature, and
x
,
x
'
are two adjacent pixels. Compared with Markov Random Field (MRF), one of advantages of CRF is that the feature functions
F
k
(
a
x
,
I
)
and
S
a
x
,
a
x
'
,
I
can use arbitrary low-level or high-level features extracted from the whole image. CRF also provides an elegant framework to combine multiple features with effective learning.”, [pg. 6, col. 1, Effectiveness of features and CRF learning, par. 1, ln. 1 to par. 2, ln. 4] “To evaluate the effectiveness of each salient object feature, we trained four CRFs: three CRFs with individual features and one CRF with all three features. Figure 10 shows the precision, recall, and F-measure of these CRFs on the image sets A and B… Figure 11 shows the feature maps and labeling results of several examples. Each feature has its own strengths and limitations. By combining all features with the pairwise feature, the CRF successfully locates the most salient object.”, [pg. 6, Figure 11, see far right]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Rahul, Vembar, and Liu as within the same field of image processing related to saliency determination, and Rahul and Liu as specifically related to the field of saliency determination for an image. The motivation to combine is disclosed in Liu, wherein Liu specifically discloses saliency as applicable to image compression ([pg. 1, col. 1, 1. Introduction, par. 1, ln. 4-8] “There are many applications for visual attention, for example, automatic image cropping [23], adaptive image display on small devices [4], image/video compression, advertising design [7], and image collection browsing”), and furthermore, would have been obvious to one of ordinary skill in the art, in that the machine learning model of Liu is far more adaptable than the static method of Rahul since it can learn multiple features on a local, regional, and global image level ([pg. 4, col. 1, 4. Salient Object Features, par. 1, ln. 1 to pg. 5, col. 2, par. 5, ln. 5] “…we introduce local, regional, and global features that define a salient object… Contrast is the most commonly used local feature for attention detection… we simply define the multiscale contrast feature
f
c
x
,
I
as a linear combination of contrasts in the Gaussian image pyramid:
f
c
x
,
I
=
∑
l
=
1
L
∑
x
'
∈
N
(
x
)
I
l
x
-
I
l
(
x
'
)
2
(9)… An example is shown in Figure 5. Multi-scale contrast highlights the high contrast boundaries by giving low scores to the homogenous regions inside the salient object… As shown in Figure 2, the salient object usually has a larger extent than local contrast and can be distinguished from its surrounding context. Therefore, we propose a regional salient feature. Suppose the salient object is enclosed by a rectangle R. We construct a surrounding contour
R
S
with the same area of R, as shown in Figure 6 (a). To measure how distinct the salient object in the rectangle is with respect to its surroundings, we can measure the distance between R and
R
S
using various visual cues such as intensity, color, and texture/texton. In this paper, we use the
χ
2
distance between histograms of RGB color:
χ
2
R
,
R
S
=
1
2
∑
(
R
i
-
R
S
i
)
2
R
i
+
R
S
i
. We use histograms because they are robust global description of appearance. They are insensitive to small changes in size, shape, and viewpoint. Another reason is that the histogram of a rectangle with any location and size can be very quickly computed by means of integral histogram introduced recently [20]. Figure 6 (a) shows that the salient object (the girl) is most distinct using the
χ
2
histogram distance… The center-surround histogram is a regional feature. Is there a global feature related to the salient object? We observe from Figure 2 that the wider a color is distributed in the image, the less possible a salient object contains this color. The global spatial distribution of a specific color can be used to describe the saliency of an object. To describe the spatial-distribution of a specific color, the simplest approach is to compute the spatial variance of the color. First, all colors in the image are represented by Gaussian Mixture Models (GMMs)
w
c
,
μ
c
,
Σ
C
c
=
1
C
, where
w
c
,
μ
c
,
Σ
C
is the weight, the mean color and the covariance matrix of the cth component. Each pixel is assigned to a color component with probability: … (12), Finally, the color spatial-distribution feature
f
s
x
,
I
is defined as a weighted sum:
f
s
x
,
I
∝
∑
c
p
c
I
x
▪
1
-
V
c
.
(15)… Figure 8 (b) shows color spatial-distribution feature maps of several example images. The salient objects are well covered by this global feature… As shown in Figure 8 (c), center-weighted, color spatial variance shows a better prediction of the saliency of each color. To verify the effectiveness of this global feature, we plot the color spatial-variance versus average saliency probability curve on the image set A, as shown in Figure 9. Obviously, the smaller a color variance is, the higher probability the color belongs to the salient object.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Rahul with the progressive image rendering and concentric regions of Vembar, and further combined the method of the combination of Rahul and Vembar with the machine learning model for saliency detection of Liu, through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Rahul with the progressive image rendering and concentric regions of Vembar and the machine learning model for saliency detection of Liu to obtain the invention as specified in claim 7.
24. Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over “Saliency enabled compression in JPEG framework” to Rahul, in view of U.S. Publication No. 2018/0288423 to Vembar, and further in view of U.S. Patent No. 6,314,452 to Dekel et al. (hereinafter Dekel).
25. Regarding Claim 23, a combination of Rahul and Vembar teaches the method of claim 1. Rahul further discloses wherein the receiving of the plurality of bytes of the encoded image occurs over a communications network ([pg. 1142, col. 1, 1 Introduction, par. 1, ln. 1-7]), and wherein the {progressively} rendering of the decoded version of the encoded image is {based on one or more network} characteristics of the communications network ([pg. 1143, col. 2, par. 1, ln. 2-10] “The required bit-rate after encoding by JPEG baseline is mainly controlled in the quantisation phase. The DCT coefficients of all the blocks are quantised by fixed quantisation parameter of its quantisation table (T, i.e. a matrix of quantisation step sizes). According to JPEG standard, the quantisation table can be configured as per the bit-rate requirement [18]. There have been series of several quantisation tables developed and are widely used for the requirement of higher CR or improved reconstructed image quality.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that the bit rate requirement is typically controlled via the communication network (i.e., number of bits that can be transmitted at any given time is a result of the network throughput, traffic, etc.). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that while Rahul discloses controlling the quantization of the image based on the bit rate requirement, Rahul does not specifically disclose wherein the image is progressively rendered based on said bit rate requirement. Therefore, Rahul fails to disclose progressively rendering based on the one or more network characteristics.
However, Vembar teaches progressively rendering an image ([Fig. 6E-L], [par. 0151, ln. 1-21], [par. 0152, ln. 1-9], [par. 0153, ln. 1-14], [par. 0175, ln. 1-23], [par. 0176, ln. 1-8]). The motivation to combine remains analogous to claim 1. One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Rahul with the progressive image rendering of Vembar through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. However, Vembar fails to specifically discloses wherein said progressive rendering is based on the one or more network characteristics. Therefore, a combination of Rahul and Vembar fails to discloses wherein the progressive rendering is based on the one or more network characteristics.
However, Dekel teaches wherein the progressive rendering is based on one or more network characteristics ([col. 20, ln. 25-46] “As ROI data is transmitted to the client 110, the rendering algorithm is performed at certain time intervals of a few seconds. At each point in time, only one rendering task is performed for any given displayed image. To ensure that progressive rendering does not become a bottleneck, two rates are measured: the data block transfer rate and the ROI rendering speed. If it predicted that the transfer will be finished before a rendering task, a small delay is inserted, such that rendering will be performed after all the data arrives. Therefore, in a slow network scenario (as the Internet often is), for almost all of the progressive rendering tasks, no delay is inserted. With the arrival of every few kilobytes of data, containing the information of a few data blocks, a rendering task visualizes the ROI at the best possible quality. In such a case the user is aware that the bottleneck of the ROI rendering is the slow network and has the option to accept the current rendering as a good enough approximation of the image and not wait for all the data to arrive.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Rahul, Vembar, and Dekel as within the same field of encoding, decoding, and rendering of images, and Rahul and Dekel as within the same fiend of JPEG images, and as analogous to the claimed invention. The motivation to combine the method of the combination of Rahul and Vembar with the network characteristic-based rendering of Dekel is disclosed in Dekel, wherein it removes bottlenecking as a result of the progressive rendering in the case the network speed is sufficient to render the entire image ([col. 20, ln. 25-46]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Rahul with the progressive image rendering of Vembar, and further combined the method of the combination of Rahul and Vembar with the network characteristic-based rendering of Dekel through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Rahul with the progressive image rendering of Vembar and the network characteristic-based rendering of Dekel to obtain the invention as specified in claim 23.
Conclusion
26. The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. See PTO-892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAULO ANDRES GARCIA whose telephone number is (703)756-5493. The examiner can normally be reached Mon-Fri, 8-4:30PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached on (571)272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAULO ANDRES GARCIA/Examiner, Art Unit 2669
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights